From owner-freebsd-fs@FreeBSD.ORG Sun May 23 14:57:41 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 650121065679 for ; Sun, 23 May 2010 14:57:41 +0000 (UTC) (envelope-from admin@kkip.pl) Received: from mainframe.kkip.pl (kkip.pl [87.105.164.78]) by mx1.freebsd.org (Postfix) with ESMTP id ABC498FC0A for ; Sun, 23 May 2010 14:57:40 +0000 (UTC) Received: from static-78-8-144-74.ssp.dialog.net.pl ([78.8.144.74]) by mainframe.kkip.pl with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.71 (FreeBSD)) (envelope-from ) id 1OGCcM-000Iv8-1H for freebsd-fs@freebsd.org; Sun, 23 May 2010 16:57:39 +0200 Message-ID: <4BF94256.6080305@kkip.pl> Date: Sun, 23 May 2010 16:57:26 +0200 From: Bartosz Stec User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.9) Gecko/20100406 Shredder/3.0.4 MIME-Version: 1.0 To: freebsd-fs@freebsd.org X-Authenticated-User: admin@kkip.pl X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Spam-Score: -7.2 X-Spam-Score-Int: -71 X-Exim-Version: 4.71 (build at 02-Feb-2010 20:10:28) X-Date: 2010-05-23 16:57:39 X-Connected-IP: 78.8.144.74:49816 X-Message-Linecount: 152 X-Body-Linecount: 142 X-Message-Size: 6352 X-Body-Size: 5947 X-Received-Count: 1 X-Recipient-Count: 1 X-Local-Recipient-Count: 1 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Cannot import raidz pool after zero fill one hdd. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 May 2010 14:57:41 -0000 Hello list. I'm not sure if it was my terrible mistake or some bug, but at this moment my filesystem is unbootable and unimportable from fixit environment and I'm pulling my hair off :(. Here's how it happened: I have a raidz pool consists of 3 HDD - ad0, ad1, ad2. It's zfs-only i386-CURRENT system created by guide http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/RAIDZ1. Smartctl has showing some warnings about offline uncorrectable sectors on ad0 every reboot, so I've decided to get rid of this by filling hdd with zeroes, rebuilding gpt table and resilver zfs partition. So what I did was: zpool offline zroot gpt/disk0 swapoff gpt/swap0 sysctl kern./geom/.debugflags=/0x10/ dd if=/dev/zero of=/dev/ad0 bs=1M After a while I've checked zpool status and ad0 was indeed offline, but with about 200 errors? Shouldn't ZFS stop using this drive at all? A moment later I tried to run bash script but I got i/o error. zpool status showed some unrecoverable error in this file and in some libraries. My guess was that zpool was still trying to use partition gpt/disk0 which is offline, so to avoid fs corruption I've shutdown my system and booted parted magic bootCD which has zerofill utility, and ad0 was filled with zeroes. After reboot loader shows some LBA errors and that it cannot find zpool. I've booted this machine using about month old freebsd-current snapshot and got into fixit environment. Here's output of zpool import: http://img715.imageshack.us/img715/7860/19843139.jpg Zroot is in UNAVAIL state because of ad2p3 OFFLINE (which was never offline), gpt/disk1 UNAVAIL, which is the same partition as ad1p3 which is showed as ONLINE. ad1p3 (gpt/disk1) and ad2p3 (gpt/disk2) should be both online and alive because I didn't do anything with them. I played with labels to correct this situation bu without success. I'm out of ideas, maybe some of you could help with this mess. Taking ad2p3 online should be enough but is it possible without importing pool? Google isn't really helpful with that case. Just to be sure if I filled correct drive with zeroes I checked output of 'head /dev/ad[0-2]p3. ad[1-2]p3 contain data so there was no mistake in there. BUT I could made a mistake in other place - I didn't checked if gpt/disk0 lays for sure on ad0 :( Maybe this is a cause of trouble - ad2 could be offline and zeroes flied to ad0 which was online. Hovewer, system should be still alive if I could revert offline state of ad2p3. Thanks in advance for help . -- Bartosz From owner-freebsd-fs@FreeBSD.ORG Sun May 23 23:14:16 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56BCA106566B for ; Sun, 23 May 2010 23:14:16 +0000 (UTC) (envelope-from james-freebsd-fs2@jrv.org) Received: from mail.jrv.org (adsl-70-243-84-13.dsl.austtx.swbell.net [70.243.84.13]) by mx1.freebsd.org (Postfix) with ESMTP id 17B738FC1A for ; Sun, 23 May 2010 23:14:15 +0000 (UTC) Received: from kremvax.housenet.jrv (kremvax.housenet.jrv [192.168.3.124]) by mail.jrv.org (8.14.3/8.14.3) with ESMTP id o4NNEC6g038362; Sun, 23 May 2010 18:14:12 -0500 (CDT) (envelope-from james-freebsd-fs2@jrv.org) Authentication-Results: mail.jrv.org; domainkeys=pass (testing) header.from=james-freebsd-fs2@jrv.org DomainKey-Signature: a=rsa-sha1; s=enigma; d=jrv.org; c=nofws; q=dns; h=message-id:date:from:user-agent:mime-version:cc:subject: references:in-reply-to:content-type:content-transfer-encoding; b=c93eip0MnWlAunB4r+E4XFKGqCYFaC87okeus2CS8ULKGfZ+atNKRjzf0BfYrcMVE IeyIuUOMg9QUb9CNqP9mH89Puy/x+90Rbfiqkf5B5bs7Jyfp2x/Qi2V2W4pWUZ+DPbj w8IrIc7hRVksJyYX6G3NOBCbOPBhOYTGV0/JKA8= Message-ID: <4BF9B6C4.40004@jrv.org> Date: Sun, 23 May 2010 18:14:12 -0500 From: "James R. Van Artsdalen" User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 References: <4BF94256.6080305@kkip.pl> In-Reply-To: <4BF94256.6080305@kkip.pl> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs Subject: Re: Cannot import raidz pool after zero fill one hdd. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 May 2010 23:14:16 -0000 Bartosz Stec wrote: > I have a raidz pool consists of 3 HDD - ad0, ad1, ad2. No, the pool is ad1p3, ad2p3, and "gpt/disk1" - what is that? Is gpt/disk1 on ad3 and not ad1? From owner-freebsd-fs@FreeBSD.ORG Mon May 24 06:13:43 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F6BD1065670; Mon, 24 May 2010 06:13:43 +0000 (UTC) (envelope-from eugene@imedia.ru) Received: from lynx.imedia.ru (lynx.imedia.ru [212.65.64.254]) by mx1.freebsd.org (Postfix) with ESMTP id 88D5C8FC19; Mon, 24 May 2010 06:13:42 +0000 (UTC) Received: from badger.imedia.ru (root@badger.imedia.ru [10.167.1.243]) by lynx.imedia.ru (8.14.3/8.14.3/TWINS7_LDAP) with ESMTP id o4O6DU6k023522; Mon, 24 May 2010 10:13:30 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from badger.imedia.ru (eugene@localhost [127.0.0.1]) by badger.imedia.ru (8.14.3/8.13.1) with ESMTP id o4O6DUXj030772; Mon, 24 May 2010 10:13:30 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from localhost (localhost [[UNIX: localhost]]) by badger.imedia.ru (8.14.3/8.13.8/Submit) id o4O6DTB3030771; Mon, 24 May 2010 10:13:29 +0400 (MSD) (envelope-from eugene@imedia.ru) From: Eugene Mitrofanov Organization: Independent Media Sanoma Magazines To: jhell Date: Mon, 24 May 2010 10:13:28 +0400 User-Agent: KMail/1.9.10 References: <201005211555.59622.eugene@imedia.ru> <4BF78226.6020403@dataix.net> In-Reply-To: <4BF78226.6020403@dataix.net> X-Origin: badger.imedia.ru MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201005241013.29344.eugene@imedia.ru> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0.1 (lynx.imedia.ru [10.167.0.252]); Mon, 24 May 2010 10:13:30 +0400 (MSD) X-Virus-Scanned: clamav-milter 0.96-exp at lynx.imedia.ru X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: FreeBSD 8.1-PRERELEASE: property 'jailed' not supported on FreeBSD: permission denied X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Eugene Mitrofanov List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2010 06:13:43 -0000 On Saturday 22 May 2010, jhell wrote: > On 05/21/2010 07:55, Eugene Mitrofanov wrote: > > Hi > > > > The command "zfs set jailed=on tank/s1" is failed with the message " > > property 'jailed' not supported on FreeBSD: permission denied". > > > > Output of "zfs get jailed tank/s1" shows me that the property "jailed" is > > still exists: > > NAME PROPERTY VALUE SOURCE > > tank/s1 jailed off default > > > > How can I change its value? > > > > Thanks. > > Simply put, > > property 'jailed' not supported on FreeBSD. > > Some features that you may see in a "zfs get all pool" will not work > because they are not implemented yet or are not planned to be > implemented because they are too *Solaris dependent. > But this feature was in 7S and in 8.0R: root@donkey:samba33# uname -sr FreeBSD 7.3-RELEASE root@donkey:samba33# zfs set jailed=on data/test root@donkey:samba33# zfs get jailed data/test NAME PROPERTY VALUE SOURCE data/test jailed on local When I updated to 8.1PRE it stopped working. Are there any plans for the revival of "jailed"? Good luck -- EMIT-RIPN, EVM7-RIPE From owner-freebsd-fs@FreeBSD.ORG Mon May 24 07:46:49 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A59581065675 for ; Mon, 24 May 2010 07:46:49 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta05.westchester.pa.mail.comcast.net (qmta05.westchester.pa.mail.comcast.net [76.96.62.48]) by mx1.freebsd.org (Postfix) with ESMTP id 65FAE8FC1F for ; Mon, 24 May 2010 07:46:49 +0000 (UTC) Received: from omta01.westchester.pa.mail.comcast.net ([76.96.62.11]) by qmta05.westchester.pa.mail.comcast.net with comcast id MKmp1e0020EZKEL55KmpFN; Mon, 24 May 2010 07:46:49 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta01.westchester.pa.mail.comcast.net with comcast id MKmn1e00B3S48mS3MKmoo7; Mon, 24 May 2010 07:46:49 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 2E9DC9B422; Mon, 24 May 2010 00:46:46 -0700 (PDT) Date: Mon, 24 May 2010 00:46:46 -0700 From: Jeremy Chadwick To: Eugene Mitrofanov Message-ID: <20100524074646.GA18394@icarus.home.lan> References: <201005211555.59622.eugene@imedia.ru> <4BF78226.6020403@dataix.net> <201005241013.29344.eugene@imedia.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201005241013.29344.eugene@imedia.ru> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, freebsd-questions@freebsd.org, trasz@freebsd.org Subject: Re: FreeBSD 8.1-PRERELEASE: property 'jailed' not supported on FreeBSD: permission denied X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2010 07:46:49 -0000 On Mon, May 24, 2010 at 10:13:28AM +0400, Eugene Mitrofanov wrote: > On Saturday 22 May 2010, jhell wrote: > > On 05/21/2010 07:55, Eugene Mitrofanov wrote: > > > Hi > > > > > > The command "zfs set jailed=on tank/s1" is failed with the message " > > > property 'jailed' not supported on FreeBSD: permission denied". > > > > > > Output of "zfs get jailed tank/s1" shows me that the property "jailed" is > > > still exists: > > > NAME PROPERTY VALUE SOURCE > > > tank/s1 jailed off default > > > > > > How can I change its value? > > > > > > Thanks. > > > > Simply put, > > > > property 'jailed' not supported on FreeBSD. > > > > Some features that you may see in a "zfs get all pool" will not work > > because they are not implemented yet or are not planned to be > > implemented because they are too *Solaris dependent. > > > > But this feature was in 7S and in 8.0R: > > root@donkey:samba33# uname -sr > FreeBSD 7.3-RELEASE > root@donkey:samba33# zfs set jailed=on data/test > root@donkey:samba33# zfs get jailed data/test > NAME PROPERTY VALUE SOURCE > data/test jailed on local > > When I updated to 8.1PRE it stopped working. Are there any plans for the > revival of "jailed"? ZFS_PROP_ZONED (property "jailed") was explicitly added to the not-supported-on-FreeBSD property list as of 5 weeks ago per MFC r197867. See commit 1.4.2.4 to RELENG_8 here: http://www.freebsd.org/cgi/cvsweb.cgi/src/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c And the piece which was committed to HEAD: http://svn.freebsd.org/viewvc/base?view=revision&revision=197867 CC'ing responsible committers to answer your question. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Mon May 24 11:06:53 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA2C81065672 for ; Mon, 24 May 2010 11:06:53 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (unknown [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B9C318FC18 for ; Mon, 24 May 2010 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o4OB6rxw004366 for ; Mon, 24 May 2010 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o4OB6rPI004364 for freebsd-fs@FreeBSD.org; Mon, 24 May 2010 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 24 May 2010 11:06:53 GMT Message-Id: <201005241106.o4OB6rPI004364@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2010 11:06:53 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server o kern/146375 fs [nfs] [patch] Typos in macro variables names in sys/fs o kern/145778 fs [zfs] [panic] panic in zfs_fuid_map_id (known issue fi s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat s kern/145424 fs [zfs] [patch] move source closer to v15 o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an o kern/145309 fs [disklabel]: Editing disk label invalidates the whole o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c o kern/144458 fs [nfs] [patch] nfsd fails as a kld p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o bin/144214 fs zfsboot fails on gang block after upgrade to zfs v14 o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o kern/143345 fs [ext2fs] [patch] extfs minor header cleanups to better o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142924 fs [ext2fs] [patch] Small cleanup for the inode struct in o kern/142914 fs [zfs] ZFS performance degradation over time o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142401 fs [ntfs] [patch] Minor updates to NTFS from NetBSD o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140134 fs [msdosfs] write and fsck destroy filesystem integrity o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs o bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/139363 fs [nfs] diskless root nfs mount from non FreeBSD server o kern/138790 fs [zfs] ZFS ceases caching when mem demand is high o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb f kern/137037 fs [zfs] [hang] zfs rollback on root causes FreeBSD to fr o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic o kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133614 fs [panic] panic: ffs_truncate: read-only filesystem o kern/133174 fs [msdosfs] [patch] msdosfs must support utf-encoded int f kern/133150 fs [zfs] Page fault with ZFS on 7.1-RELEASE/amd64 while w o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130979 fs [smbfs] [panic] boot/kernel/smbfs.ko o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130229 fs [iconv] usermount fails on fs that need iconv o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129059 fs [zfs] [patch] ZFS bootloader whitelistable via WITHOUT f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS p kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition f bin/124424 fs [zfs] zfs(8): zfs list -r shows strange snapshots' siz o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121779 fs [ufs] snapinfo(8) (and related tools?) only work for t o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha f kern/120991 fs [panic] [fs] [snapshot] System crashes when manipulati o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F f kern/119735 fs [zfs] geli + ZFS + samba starting on boot panics 7.0-B o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs mv(1): moving a directory changes its mtime o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116913 fs [ffs] [panic] ffs_blkfree: freeing free block p kern/116608 fs [msdosfs] [patch] msdosfs fails to check mount options o kern/116583 fs [ffs] [hang] System freezes for short time when using o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/115645 fs [snapshots] [panic] lockmgr: thread 0xc4c00d80, not ex o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] mount_msdosfs: msdosfs_iconv: Operation not o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/106030 fs [ufs] [panic] panic in ufs from geom when a dead disk o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [iso9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna f kern/91568 fs [ufs] [panic] writing to UFS/softupdates DVD media in o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o kern/85326 fs [smbfs] [panic] saving a file via samba to an overquot o kern/84589 fs [2TB] 5.4-STABLE unresponsive during background fsck 2 o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/53137 fs [ffs] [panic] background fscking causing ffs_valloc pa o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o kern/33464 fs [ufs] soft update inconsistencies after system crash o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 174 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon May 24 12:31:11 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 982F9106568B; Mon, 24 May 2010 12:31:11 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 380748FC2B; Mon, 24 May 2010 12:31:10 +0000 (UTC) Received: by gyh20 with SMTP id 20so1917973gyh.13 for ; Mon, 24 May 2010 05:31:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:openpgp:content-type:content-transfer-encoding; bh=R0xFEUEObIvvhd+izsFOEAvtO+l8KW5vV9lkiYgzA5M=; b=krSfhsWnKlTLyj5E9cTTopEurdyQyIqJaurWvN1SnW+ua6wxnQejafMJC9o8tm0dpO 9+QXteT9kzX2UV0bWIdRF3SA/sEFXcJXUP9Dn/XoiDVT+WovIcNJr2eqD0NeOAIXw3BC tD7hwd9rHxvs4YYjd1ScMfEays88OKiOqTVcg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:openpgp:content-type :content-transfer-encoding; b=GNgWS6k1QOs0KkmQ5//LpwVe31r0eeidOGyPZQESEUKg3ZmUJwDMq+UWt3LHFNBUQO Yeks5Hxrru9rJ2S52SsT1iSswHMHU69j06VhtkawLQQ6vn0brN/Dp02phnTSJ+ElgIaD JNVEpxi9btE8dLEMn9eO1rBmsjWStNgqxqYhQ= Received: by 10.151.25.16 with SMTP id c16mr6063916ybj.363.1274704270454; Mon, 24 May 2010 05:31:10 -0700 (PDT) Received: from centel.dataix.local (adsl-99-19-40-41.dsl.klmzmi.sbcglobal.net [99.19.40.41]) by mx.google.com with ESMTPS id 20sm3027057ywh.11.2010.05.24.05.31.09 (version=SSLv3 cipher=RC4-MD5); Mon, 24 May 2010 05:31:09 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4BFA718C.3020202@dataix.net> Date: Mon, 24 May 2010 08:31:08 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100515 Thunderbird MIME-Version: 1.0 To: Eugene Mitrofanov References: <201005211555.59622.eugene@imedia.ru> <4BF78226.6020403@dataix.net> <201005241013.29344.eugene@imedia.ru> In-Reply-To: <201005241013.29344.eugene@imedia.ru> X-Enigmail-Version: 1.0.1 OpenPGP: id=89D8547E Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: FreeBSD 8.1-PRERELEASE: property 'jailed' not supported on FreeBSD: permission denied X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2010 12:31:11 -0000 On 05/24/2010 02:13, Eugene Mitrofanov wrote: > On Saturday 22 May 2010, jhell wrote: >> On 05/21/2010 07:55, Eugene Mitrofanov wrote: >>> Hi >>> >>> The command "zfs set jailed=on tank/s1" is failed with the message " >>> property 'jailed' not supported on FreeBSD: permission denied". >>> >>> Output of "zfs get jailed tank/s1" shows me that the property "jailed" is >>> still exists: >>> NAME PROPERTY VALUE SOURCE >>> tank/s1 jailed off default >>> >>> How can I change its value? >>> >>> Thanks. >> >> Simply put, >> >> property 'jailed' not supported on FreeBSD. >> >> Some features that you may see in a "zfs get all pool" will not work >> because they are not implemented yet or are not planned to be >> implemented because they are too *Solaris dependent. >> > > But this feature was in 7S and in 8.0R: > > root@donkey:samba33# uname -sr > FreeBSD 7.3-RELEASE > root@donkey:samba33# zfs set jailed=on data/test > root@donkey:samba33# zfs get jailed data/test > NAME PROPERTY VALUE SOURCE > data/test jailed on local > > When I updated to 8.1PRE it stopped working. Are there any plans for the > revival of "jailed"? > > Good luck And what exactly did that property do for you... ?||? AFAIK it was a NOP. -- jhell From owner-freebsd-fs@FreeBSD.ORG Mon May 24 13:06:44 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12B67106566C; Mon, 24 May 2010 13:06:44 +0000 (UTC) (envelope-from eugene@imedia.ru) Received: from lynx.imedia.ru (lynx.imedia.ru [212.65.64.254]) by mx1.freebsd.org (Postfix) with ESMTP id 6357A8FC14; Mon, 24 May 2010 13:06:42 +0000 (UTC) Received: from badger.imedia.ru (root@badger.imedia.ru [10.167.1.243]) by lynx.imedia.ru (8.14.3/8.14.3/TWINS7_LDAP) with ESMTP id o4OD6boi023056; Mon, 24 May 2010 17:06:38 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from badger.imedia.ru (eugene@localhost [127.0.0.1]) by badger.imedia.ru (8.14.3/8.13.1) with ESMTP id o4OD6bdu035284; Mon, 24 May 2010 17:06:37 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from localhost (localhost [[UNIX: localhost]]) by badger.imedia.ru (8.14.3/8.13.8/Submit) id o4OD6bie035283; Mon, 24 May 2010 17:06:37 +0400 (MSD) (envelope-from eugene@imedia.ru) From: Eugene Mitrofanov Organization: Independent Media Sanoma Magazines To: jhell Date: Mon, 24 May 2010 17:06:37 +0400 User-Agent: KMail/1.9.10 References: <201005211555.59622.eugene@imedia.ru> <201005241013.29344.eugene@imedia.ru> <4BFA718C.3020202@dataix.net> In-Reply-To: <4BFA718C.3020202@dataix.net> X-Origin: badger.imedia.ru MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201005241706.37396.eugene@imedia.ru> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0.1 (lynx.imedia.ru [10.167.0.252]); Mon, 24 May 2010 17:06:38 +0400 (MSD) X-Virus-Scanned: clamav-milter 0.96-exp at lynx.imedia.ru X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: FreeBSD 8.1-PRERELEASE: property 'jailed' not supported on FreeBSD: permission denied X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Eugene Mitrofanov List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2010 13:06:44 -0000 On Monday 24 May 2010, jhell wrote: > On 05/24/2010 02:13, Eugene Mitrofanov wrote: > > On Saturday 22 May 2010, jhell wrote: > >> On 05/21/2010 07:55, Eugene Mitrofanov wrote: > >>> Hi > >>> > >>> The command "zfs set jailed=on tank/s1" is failed with the message " > >>> property 'jailed' not supported on FreeBSD: permission denied". > >>> > >>> Output of "zfs get jailed tank/s1" shows me that the property "jailed" is > >>> still exists: > >>> NAME PROPERTY VALUE SOURCE > >>> tank/s1 jailed off default > >>> > >>> How can I change its value? > >>> > >>> Thanks. > >> > >> Simply put, > >> > >> property 'jailed' not supported on FreeBSD. > >> > >> Some features that you may see in a "zfs get all pool" will not work > >> because they are not implemented yet or are not planned to be > >> implemented because they are too *Solaris dependent. > >> > > > > But this feature was in 7S and in 8.0R: > > > > root@donkey:samba33# uname -sr > > FreeBSD 7.3-RELEASE > > root@donkey:samba33# zfs set jailed=on data/test > > root@donkey:samba33# zfs get jailed data/test > > NAME PROPERTY VALUE SOURCE > > data/test jailed on local > > > > When I updated to 8.1PRE it stopped working. Are there any plans for the > > revival of "jailed"? > > > > Good luck > > And what exactly did that property do for you... ?||? AFAIK it was a NOP. > > -- > > jhell > > I want to set up something like described in http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2009-12/msg00028.html -- EMIT-RIPN, EVM7-RIPE From owner-freebsd-fs@FreeBSD.ORG Mon May 24 15:27:06 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57D7B106567D for ; Mon, 24 May 2010 15:27:06 +0000 (UTC) (envelope-from james-freebsd-fs2@jrv.org) Received: from zimbra.jrv.org (rrcs-24-73-246-106.sw.biz.rr.com [24.73.246.106]) by mx1.freebsd.org (Postfix) with ESMTP id 2F8C58FC18 for ; Mon, 24 May 2010 15:27:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by zimbra.jrv.org (Postfix) with ESMTP id 1672C16A055 for ; Mon, 24 May 2010 10:27:05 -0500 (CDT) X-Virus-Scanned: amavisd-new at zimbra.housenet.jrv Received: from zimbra.jrv.org ([127.0.0.1]) by localhost (zimbra.housenet.jrv [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FGN0ZfUjbDjV for ; Mon, 24 May 2010 10:27:04 -0500 (CDT) Received: from [10.0.2.15] (adsl-70-243-84-14.dsl.austtx.swbell.net [70.243.84.14]) by zimbra.jrv.org (Postfix) with ESMTPSA id 61E2416A048 for ; Mon, 24 May 2010 10:27:04 -0500 (CDT) Message-ID: <4BFA9AEC.1070608@jrv.org> Date: Mon, 24 May 2010 10:27:40 -0500 From: "James R. Van Artsdalen" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: ZFS panic: wrong length 131072 for sectorsize 2352 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2010 15:27:06 -0000 amd64, svn 208373, in VirtualBox with no disc in the virtual CD drive. ... Trying to mount root from zfs:CLANK vdev_geom_open_by_guid:362[1]: Searching by guid [1976847806106852025]. vdev_geom_read_guid:230[1]: Reading guid from acd0t01... panic: wrong length 131072 for sectorsize 2352 cpuid = 1 KDB: enter: panic [ thread pid 2 tid 100009 ] Stopped at kdb_enter+0x3d: movq $0,0x6cc120(%rip) db> bt bt Tracing pid 2 tid 100009 td 0xffffff00028b8b40 kdb_enter() at kdb_enter+0x3d panic() at panic+0x17b g_io_request() at g_io_request+0x2a0 vdev_geom_read_guid() at vdev_geom_read_guid+0x226 vdev_geom_attach_by_guid_event() at vdev_geom_attach_by_guid_event+0x155 g_run_events() at g_run_events+0x217 g_event_procbody() at g_event_procbody+0x6c fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800005cd30, rbp = 0 --- db> Tracing pid 2 tid 100009 td 0xffffff00028b8b40 kdb_enter() at kdb_enter+0x3d panic() at panic+0x17b g_io_request() at g_io_request+0x2a0 vdev_geom_read_guid() at vdev_geom_read_guid+0x226 vdev_geom_attach_by_guid_event() at vdev_geom_attach_by_guid_event+0x155 g_run_events() at g_run_events+0x217 g_event_procbody() at g_event_procbody+0x6c fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800005cd30, rbp = 0 --- db> From owner-freebsd-fs@FreeBSD.ORG Mon May 24 15:50:50 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68276106566C; Mon, 24 May 2010 15:50:50 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id C6A3E8FC12; Mon, 24 May 2010 15:50:49 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEAMo8+kuDaFvH/2dsb2JhbACeC3HAGIJuCIIdBA X-IronPort-AV: E=Sophos;i="4.53,292,1272859200"; d="scan'208";a="77489890" Received: from danube.cs.uoguelph.ca ([131.104.91.199]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 24 May 2010 11:50:47 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by danube.cs.uoguelph.ca (Postfix) with ESMTP id 0F8C41084123; Mon, 24 May 2010 11:50:48 -0400 (EDT) X-Virus-Scanned: amavisd-new at danube.cs.uoguelph.ca Received: from danube.cs.uoguelph.ca ([127.0.0.1]) by localhost (danube.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RL43UYd4YmHN; Mon, 24 May 2010 11:50:46 -0400 (EDT) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by danube.cs.uoguelph.ca (Postfix) with ESMTP id D284A10840B6; Mon, 24 May 2010 11:50:46 -0400 (EDT) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id o4OG6HD02756; Mon, 24 May 2010 12:06:17 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Mon, 24 May 2010 12:06:16 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: John Baldwin In-Reply-To: <201005211039.04108.jhb@freebsd.org> Message-ID: References: <201005191144.00382.jhb@freebsd.org> <201005200922.17245.jhb@freebsd.org> <201005211039.04108.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Rick Macklem , Robert Watson , fs@freebsd.org Subject: Re: [PATCH] Better handling of stale filehandles in open() in the NFS client X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2010 15:50:50 -0000 On Fri, 21 May 2010, John Baldwin wrote: > No, it's not the lock, it's this thing Mohan added here in nfs_open(): > > struct thread *td = curthread; > > if (np->n_ac_ts_syscalls != td->td_syscalls || > np->n_ac_ts_tid != td->td_tid || > td->td_proc == NULL || > np->n_ac_ts_pid != td->td_proc->p_pid) { > np->n_attrstamp = 0; > KDTRACE_NFS_ATTRCACHE_FLUSH_DONE(vp); > } > > Which used to be an unconditional clearing of n_attrstamp so that the > VOP_GETATTR() in nfs_open() would always go over the wire. Now it doesn't > clear the attributes if the attribute cache was last updated during the same > system call that is invoking nfs_open() by the same thread. > > Hmm, however, concurrent lookups of the same pathname may cause this test to > fail and still clear n_attrstamp since we now use shared vnode locks for > pathname lookups. That was true before my change as well. In fact, using > shared vnode locks for read-only opens (the MNTK_EXTENDED_SHARED flag) > probably does cause Mohan's patch to not work in many cases which probably > accounts for the small increase in RPC counts. > > Try setting the vfs.lookup_shared sysctl to 0 to see if that removes all the > extra RPCs. Another change would be to readd the change to flush the > attribute cache on close and see if that also brings back extra > attribute/access RPCs without my patch (but with lookup_shared left enabled) > to verify that shared lookups are the root cause of extra RPCs. > Ok, I've played with this a bit and here's what I've seen: - Disabling vfs.lookup_shared had no effect. Since I'm running a single thread (no "-j" on the make) ad nothing else on the machine during the tests, that seems to make sense. - When I added some printfs, here's what seems to be happening. snippet of nfs_lookup(): if ((cnp->cn_flags & (ISLASTCN | ISOPEN)) == (ISLASTCN | ISOPEN)) { newnp = VTONFS(newvp); mtx_lock(&newnp->n_mtx); newnp->n_attrstamp = 0; mtx_unlock(&newnp->n_mtx); } if (!VOP_GETATTR(newvp, &vattr, cnp->cn_cred) && vattr.va_ctime.tv_sec == newnp->n_ctime) { nfsstats.lookupcache_hits++; if (cnp->cn_nameiop != LOOKUP && (flags & ISLASTCN)) cnp->cn_flags |= SAVENAME; return (0); } --> gets here about 9,000 times --> As far as I can see, these getattrs are part of the price you pay for your patch. (Cache misses that aren't caused by VOP_GETATTR() failing.) snippet of nfs_open(): if (np->n_flag & NMODIFIED) { ... --> gets here about 6,000 times. The extra Getattrs for this case can be avoied by adding if ((newnp->n_flag & NMODIFIED) == 0) for setting newnp->n_attrstamp = 0; above. } else { struct thread *td = curthread; if (np->n_ac_ts_syscalls != td->td_syscalls || np->n_ac_ts_tid != td->td_tid || td->td_proc == NULL || np->n_ac_ts_pid != td->td_proc->p_pid) { np->n_attrstamp = 0; KDTRACE_NFS_ATTRCACHE_FLUSH_DONE(vp); --> never gets here, which makes sense since your patch forces the Getattr to be done in nfs_lookup(). I wouldn't say that this can "never" happen with the patch applied, because of concurrency with shared vnode locks, but it isn't happening for this test. } mtx_unlock(&np->n_mtx); error = VOP_GETATTR(vp, &vattr, ap->a_cred); So, I can see where about 15,000 of the extra Getattrs occur, but haven't nailed down the other 20,000. I do see a lot of cases where the namei() call in vn_open_cred() at line#189 of vfs_vnops.c returns an error. I haven't figured out if those account for the other 20,000. (ie. are there cases where namei()/nfs_lookup() fails after having forced the Getattr) Since namei() fails, VOP_OPEN() doesn't get called. In summary, I haven't nailed down where 20,000 of the extra Getattr RPCs happen, but if it still achieves what you want, adding the if ((newnp->n_flag & NMODIFIED) == 0) before newnp->n_attrstamp = 0; gets rid of some of them. I'm going to run some tests with Readdirplus enabled, to see what effect the patch has for that case, rick. From owner-freebsd-fs@FreeBSD.ORG Tue May 25 08:19:56 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C104106566C for ; Tue, 25 May 2010 08:19:56 +0000 (UTC) (envelope-from james-freebsd-fs2@jrv.org) Received: from zimbra.jrv.org (adsl-70-243-84-11.dsl.austtx.swbell.net [70.243.84.11]) by mx1.freebsd.org (Postfix) with ESMTP id 1DF4C8FC12 for ; Tue, 25 May 2010 08:19:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by zimbra.jrv.org (Postfix) with ESMTP id ACCC116A059 for ; Tue, 25 May 2010 03:19:54 -0500 (CDT) X-Virus-Scanned: amavisd-new at zimbra.housenet.jrv Received: from zimbra.jrv.org ([127.0.0.1]) by localhost (zimbra.housenet.jrv [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mUTKKlDP4hBP for ; Tue, 25 May 2010 03:19:54 -0500 (CDT) Received: from [10.0.2.15] (adsl-70-243-84-14.dsl.austtx.swbell.net [70.243.84.14]) by zimbra.jrv.org (Postfix) with ESMTPSA id 3863A16A04A for ; Tue, 25 May 2010 03:19:54 -0500 (CDT) Message-ID: <4BFB884A.3010804@jrv.org> Date: Tue, 25 May 2010 03:20:26 -0500 From: "James R. Van Artsdalen" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 CC: freebsd-fs@freebsd.org References: <4BFA9AEC.1070608@jrv.org> In-Reply-To: <4BFA9AEC.1070608@jrv.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: ZFS panic: wrong length 131072 for sectorsize 2352 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2010 08:19:56 -0000 On 5/24/2010 10:27 AM, James R. Van Artsdalen wrote: > panic: wrong length 131072 for sectorsize 2352 This fixes the bug. Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision 208373) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working copy) @@ -250,6 +250,9 @@ if ((offset % pp->sectorsize) != 0) continue; + if ((size % pp->sectorsize) != 0) + continue; + if (vdev_geom_io(cp, BIO_READ, label, offset, size) != 0) continue; buf = label->vl_vdev_phys.vp_nvlist; From owner-freebsd-fs@FreeBSD.ORG Tue May 25 08:35:23 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08E881065672 for ; Tue, 25 May 2010 08:35:23 +0000 (UTC) (envelope-from eugene@imedia.ru) Received: from lynx.imedia.ru (lynx.imedia.ru [212.65.64.254]) by mx1.freebsd.org (Postfix) with ESMTP id 707F58FC08 for ; Tue, 25 May 2010 08:35:21 +0000 (UTC) Received: from badger.imedia.ru (root@badger.imedia.ru [10.167.1.243]) by lynx.imedia.ru (8.14.3/8.14.3/TWINS7_LDAP) with ESMTP id o4P8ZKue027855; Tue, 25 May 2010 12:35:20 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from badger.imedia.ru (eugene@localhost [127.0.0.1]) by badger.imedia.ru (8.14.3/8.13.1) with ESMTP id o4P8ZKWY045146; Tue, 25 May 2010 12:35:20 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from localhost (localhost [[UNIX: localhost]]) by badger.imedia.ru (8.14.3/8.13.8/Submit) id o4P8ZJW2045145; Tue, 25 May 2010 12:35:19 +0400 (MSD) (envelope-from eugene@imedia.ru) From: Eugene Mitrofanov Organization: Independent Media Sanoma Magazines To: freebsd-stable@freebsd.org, freebsd-fs@freebsd.org Date: Tue, 25 May 2010 12:35:19 +0400 User-Agent: KMail/1.9.10 X-Origin: badger.imedia.ru MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201005251235.19833.eugene@imedia.ru> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0.1 (lynx.imedia.ru [10.167.0.252]); Tue, 25 May 2010 12:35:20 +0400 (MSD) X-Virus-Scanned: clamav-milter 0.96-exp at lynx.imedia.ru X-Virus-Status: Clean Cc: Subject: FreeBSD 8.1 prerelease "security.jail.mount_allowed" is broken? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Eugene Mitrofanov List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2010 08:35:23 -0000 Hello I try to do mount from a jail but it failed. Could you advise me where is my mistake? root@ftp:eugene# uname -mrs FreeBSD 8.1-PRERELEASE amd64 root@ftp:eugene# sysctl -a | grep -E '(jailed|mount)' vfs.usermount: 1 vfs.ffs.compute_summary_at_mount: 0 security.jail.mount_allowed: 1 security.jail.jailed: 1 root@ftp:eugene# mount /dev/da2s2a /var/t mount: /dev/da2s2a : Operation not permitted root@ftp:eugene# mount /dev/md1 /var/t mount: /dev/md1 : Operation not permitted root@ftp:eugene# mount /dev/zvol/tank/ftp.journal /var/t mount: /dev/zvol/tank/ftp.journal : Operation not permitted Best regards -- EMIT-RIPN, EVM7-RIPE From owner-freebsd-fs@FreeBSD.ORG Tue May 25 13:22:09 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C33A0106566B; Tue, 25 May 2010 13:22:09 +0000 (UTC) (envelope-from md@scoutsengidsenvlaanderen.be) Received: from scoutsengidsenvlaanderen.be (mail.scoutsengidsenvlaanderen.be [81.83.17.184]) by mx1.freebsd.org (Postfix) with ESMTP id 6D64B8FC0A; Tue, 25 May 2010 13:22:08 +0000 (UTC) Received: from [192.168.1.163] (account md@scoutsengidsenvlaanderen.be [192.168.1.163] verified) by scoutsengidsenvlaanderen.be (CommuniGate Pro SMTP 5.3.5) with ESMTPA id 34583959; Tue, 25 May 2010 15:22:06 +0200 Message-ID: <4BFBCEFE.30004@scoutsengidsenvlaanderen.be> Date: Tue, 25 May 2010 15:22:06 +0200 From: Michiel Detailleur User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4B97A963.9040300@scoutsengidsenvlaanderen.be> <20100310174411.GE1715@garage.freebsd.pl> <4B98E95B.90806@scoutsengidsenvlaanderen.be> <20100311130807.GJ2350@garage.freebsd.pl> In-Reply-To: <20100311130807.GJ2350@garage.freebsd.pl> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: [Spam? ] Re: ZFS snapshot name length limit? (File name too long) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2010 13:22:09 -0000 Pawel Jakub Dawidek schreef: > On Thu, Mar 11, 2010 at 02:00:11PM +0100, Michiel Detailleur wrote: > >> >From looking at the code, I think you hitting this limit: >> >>> This is FreeBSD limit caused by statfs structure: >>> >>> /* >>> * filesystem statistics >>> */ >>> [...] >>> #define MNAMELEN 88 /* size of on/from name bufs */ >>> [...] >>> >>> ZFS mounts snapshots on lookup and this is this mount that fails. >>> >>> >> So all that would need to be done to allow mounting snapshots with >> longer names would be to increase this value? (I'm not a C programmer, >> so bear with me :) ) >> > > Such change will break ABI compatibility with tools compiled on previous > FreeBSD versions. As you can see in sys/sys/mount.h this is the reason > we still keep ostatfs structure. I'll discuss the possibilities with > other FreeBSD committers and we will see what we can do about it. > Hi Pawel, Did you and the other committers have a chance to discuss this yet? It would be nice if a fix for this could make it into 8.1 :) Michiel From owner-freebsd-fs@FreeBSD.ORG Tue May 25 19:10:03 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A7DE106564A; Tue, 25 May 2010 19:10:03 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 9B9578FC08; Tue, 25 May 2010 19:10:02 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 9B88545DD8; Tue, 25 May 2010 21:10:00 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id AC8F445CBA; Tue, 25 May 2010 21:09:55 +0200 (CEST) Date: Tue, 25 May 2010 21:09:42 +0200 From: Pawel Jakub Dawidek To: Eugene Mitrofanov Message-ID: <20100525190942.GD1659@garage.freebsd.pl> References: <201005251235.19833.eugene@imedia.ru> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0QFb0wBpEddLcDHQ" Content-Disposition: inline In-Reply-To: <201005251235.19833.eugene@imedia.ru> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: FreeBSD 8.1 prerelease "security.jail.mount_allowed" is broken? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2010 19:10:03 -0000 --0QFb0wBpEddLcDHQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 25, 2010 at 12:35:19PM +0400, Eugene Mitrofanov wrote: > Hello >=20 > I try to do mount from a jail but it failed. Could you advise me where is= my=20 > mistake? >=20 > root@ftp:eugene# uname -mrs > FreeBSD 8.1-PRERELEASE amd64 > root@ftp:eugene# sysctl -a | grep -E '(jailed|mount)' > vfs.usermount: 1 > vfs.ffs.compute_summary_at_mount: 0 > security.jail.mount_allowed: 1 > security.jail.jailed: 1 > root@ftp:eugene# mount /dev/da2s2a /var/t > mount: /dev/da2s2a : Operation not permitted > root@ftp:eugene# mount /dev/md1 /var/t > mount: /dev/md1 : Operation not permitted > root@ftp:eugene# mount /dev/zvol/tank/ftp.journal /var/t > mount: /dev/zvol/tank/ftp.journal : Operation not permitted You can only mount jail-friendly file systems - those with 'jail' keyword in lsvfs(1) output. What you tried can't be safe. Imagine creating corrupted file system on da2s2a and mounting it. It will panic entire system, not only your jail. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --0QFb0wBpEddLcDHQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkv8IHYACgkQForvXbEpPzSHpACeKp6iYeGd6h/zkpoZJTIx5j9I 8S8AniB9XxU4Sr3aT8NZHdii/CFLB+0N =cdSt -----END PGP SIGNATURE----- --0QFb0wBpEddLcDHQ-- From owner-freebsd-fs@FreeBSD.ORG Tue May 25 19:27:41 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6BB431065677 for ; Tue, 25 May 2010 19:27:41 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id AF13F8FC1A for ; Tue, 25 May 2010 19:27:40 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4845945E11; Tue, 25 May 2010 21:27:39 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 4701945C9F; Tue, 25 May 2010 21:27:34 +0200 (CEST) Date: Tue, 25 May 2010 21:27:21 +0200 From: Pawel Jakub Dawidek To: "James R. Van Artsdalen" Message-ID: <20100525192721.GE1659@garage.freebsd.pl> References: <4BFA9AEC.1070608@jrv.org> <4BFB884A.3010804@jrv.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3oCie2+XPXTnK5a5" Content-Disposition: inline In-Reply-To: <4BFB884A.3010804@jrv.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS panic: wrong length 131072 for sectorsize 2352 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2010 19:27:41 -0000 --3oCie2+XPXTnK5a5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 25, 2010 at 03:20:26AM -0500, James R. Van Artsdalen wrote: > On 5/24/2010 10:27 AM, James R. Van Artsdalen wrote: > > panic: wrong length 131072 for sectorsize 2352 >=20 > This fixes the bug. I don't think so:) size should be properly calculated at this point and should be multiple of sectorsize. The problem is that vdev_geom_io() splits request into MAXPHYS chunks if it is too big, which is wrong, because MAXPHYS doesn't have to be multiple of sectorsize. Could you try this patch instead: http://people.freebsd.org/~pjd/patches/vdev_geom.c.4.patch > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision= 208373) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working = copy) > @@ -250,6 +250,9 @@ > if ((offset % pp->sectorsize) !=3D 0) > continue; >=20 > + if ((size % pp->sectorsize) !=3D 0) > + continue; > + > if (vdev_geom_io(cp, BIO_READ, label, offset, size) !=3D = 0) > continue; > buf =3D label->vl_vdev_phys.vp_nvlist; --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --3oCie2+XPXTnK5a5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkv8JJkACgkQForvXbEpPzSBcwCdF8i1d0sT07du0WGhXYpkrEFs SjoAoNwfWGkMNgVs+oKRFc7lPZyaim6N =Nx3D -----END PGP SIGNATURE----- --3oCie2+XPXTnK5a5-- From owner-freebsd-fs@FreeBSD.ORG Wed May 26 06:23:25 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6582B1065673; Wed, 26 May 2010 06:23:25 +0000 (UTC) (envelope-from eugene@imedia.ru) Received: from lynx.imedia.ru (lynx.imedia.ru [212.65.64.254]) by mx1.freebsd.org (Postfix) with ESMTP id ADC9A8FC1A; Wed, 26 May 2010 06:23:24 +0000 (UTC) Received: from badger.imedia.ru (root@badger.imedia.ru [10.167.1.243]) by lynx.imedia.ru (8.14.3/8.14.3/TWINS7_LDAP) with ESMTP id o4Q6NMpE079157; Wed, 26 May 2010 10:23:22 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from badger.imedia.ru (eugene@localhost [127.0.0.1]) by badger.imedia.ru (8.14.3/8.13.1) with ESMTP id o4Q6NMIp066844; Wed, 26 May 2010 10:23:22 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from localhost (localhost [[UNIX: localhost]]) by badger.imedia.ru (8.14.3/8.13.8/Submit) id o4Q6NMSQ066843; Wed, 26 May 2010 10:23:22 +0400 (MSD) (envelope-from eugene@imedia.ru) From: Eugene Mitrofanov Organization: Independent Media Sanoma Magazines To: Pawel Jakub Dawidek Date: Wed, 26 May 2010 10:23:21 +0400 User-Agent: KMail/1.9.10 References: <201005251235.19833.eugene@imedia.ru> <20100525190942.GD1659@garage.freebsd.pl> In-Reply-To: <20100525190942.GD1659@garage.freebsd.pl> X-Origin: badger.imedia.ru MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201005261023.22291.eugene@imedia.ru> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0.1 (lynx.imedia.ru [10.167.0.252]); Wed, 26 May 2010 10:23:22 +0400 (MSD) X-Virus-Scanned: clamav-milter 0.96-exp at lynx.imedia.ru X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: FreeBSD 8.1 prerelease "security.jail.mount_allowed" is broken? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Eugene Mitrofanov List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2010 06:23:25 -0000 On Tuesday 25 May 2010, Pawel Jakub Dawidek wrote: > On Tue, May 25, 2010 at 12:35:19PM +0400, Eugene Mitrofanov wrote: > > Hello > > > > I try to do mount from a jail but it failed. Could you advise me where is my > > mistake? > > > > root@ftp:eugene# uname -mrs > > FreeBSD 8.1-PRERELEASE amd64 > > root@ftp:eugene# sysctl -a | grep -E '(jailed|mount)' > > vfs.usermount: 1 > > vfs.ffs.compute_summary_at_mount: 0 > > security.jail.mount_allowed: 1 > > security.jail.jailed: 1 > > root@ftp:eugene# mount /dev/da2s2a /var/t > > mount: /dev/da2s2a : Operation not permitted > > root@ftp:eugene# mount /dev/md1 /var/t > > mount: /dev/md1 : Operation not permitted > > root@ftp:eugene# mount /dev/zvol/tank/ftp.journal /var/t > > mount: /dev/zvol/tank/ftp.journal : Operation not permitted > > You can only mount jail-friendly file systems - those with 'jail' > keyword in lsvfs(1) output. Unfortunately, it seems for me that 'zfs mount' is also broken in 8.1PRE (zpool ver 14). "zfs jail 4 tank" is executing successfully but the word 'jail' does not meet in the 'man zfs' anymore and 'zfs set jailed=on tank' is failed with the error "property 'jailed' not supported on FreeBSD: permission denied". "zfs mount" from jail also failed: root@ftp:eugene# sysctl security.jail.jailed security.jail.jailed: 1 root@ftp:eugene# zfs mount tank/test cannot mount 'tank/test': permission denied > What you tried can't be safe. Imagine creating corrupted file system on > da2s2a and mounting it. It will panic entire system, not only your jail. -- EMIT-RIPN, EVM7-RIPE From owner-freebsd-fs@FreeBSD.ORG Wed May 26 11:15:44 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A57D6106564A; Wed, 26 May 2010 11:15:44 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B62528FC1A; Wed, 26 May 2010 11:15:43 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA22256; Wed, 26 May 2010 14:15:41 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4BFD02DC.1050203@icyb.net.ua> Date: Wed, 26 May 2010 14:15:40 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100517) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4BFA9AEC.1070608@jrv.org> <4BFB884A.3010804@jrv.org> <20100525192721.GE1659@garage.freebsd.pl> In-Reply-To: <20100525192721.GE1659@garage.freebsd.pl> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=KOI8-U Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: ZFS panic: wrong length 131072 for sectorsize 2352 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2010 11:15:44 -0000 on 25/05/2010 22:27 Pawel Jakub Dawidek said the following: > On Tue, May 25, 2010 at 03:20:26AM -0500, James R. Van Artsdalen wrote: >> On 5/24/2010 10:27 AM, James R. Van Artsdalen wrote: >>> panic: wrong length 131072 for sectorsize 2352 >> This fixes the bug. > > I don't think so:) size should be properly calculated at this point and > should be multiple of sectorsize. The problem is that vdev_geom_io() > splits request into MAXPHYS chunks if it is too big, which is wrong, > because MAXPHYS doesn't have to be multiple of sectorsize. But do we really want to support here sector sizes that are not power of two? And/or MAXPHYS which is not power of two? > Could you try this patch instead: > > http://people.freebsd.org/~pjd/patches/vdev_geom.c.4.patch Otherwise MAXPHYS % cp->provider->sectorsize would always be zero. >> Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c >> =================================================================== >> --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision 208373) >> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working copy) >> @@ -250,6 +250,9 @@ >> if ((offset % pp->sectorsize) != 0) >> continue; >> >> + if ((size % pp->sectorsize) != 0) >> + continue; >> + >> if (vdev_geom_io(cp, BIO_READ, label, offset, size) != 0) >> continue; >> buf = label->vl_vdev_phys.vp_nvlist; > -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Wed May 26 11:49:30 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7553F106566C for ; Wed, 26 May 2010 11:49:30 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id B7A598FC08 for ; Wed, 26 May 2010 11:49:29 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 0029245CA0; Wed, 26 May 2010 13:49:26 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 243A64569A; Wed, 26 May 2010 13:49:22 +0200 (CEST) Date: Wed, 26 May 2010 13:49:09 +0200 From: Pawel Jakub Dawidek To: Andriy Gapon Message-ID: <20100526114909.GC3339@garage.freebsd.pl> References: <4BFA9AEC.1070608@jrv.org> <4BFB884A.3010804@jrv.org> <20100525192721.GE1659@garage.freebsd.pl> <4BFD02DC.1050203@icyb.net.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NKoe5XOeduwbEQHU" Content-Disposition: inline In-Reply-To: <4BFD02DC.1050203@icyb.net.ua> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org Subject: Re: ZFS panic: wrong length 131072 for sectorsize 2352 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2010 11:49:30 -0000 --NKoe5XOeduwbEQHU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 26, 2010 at 02:15:40PM +0300, Andriy Gapon wrote: > on 25/05/2010 22:27 Pawel Jakub Dawidek said the following: > > On Tue, May 25, 2010 at 03:20:26AM -0500, James R. Van Artsdalen wrote: > >> On 5/24/2010 10:27 AM, James R. Van Artsdalen wrote: > >>> panic: wrong length 131072 for sectorsize 2352 > >> This fixes the bug. > >=20 > > I don't think so:) size should be properly calculated at this point and > > should be multiple of sectorsize. The problem is that vdev_geom_io() > > splits request into MAXPHYS chunks if it is too big, which is wrong, > > because MAXPHYS doesn't have to be multiple of sectorsize. >=20 > But do we really want to support here sector sizes that are not power of = two? We can easly support them, so why not? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --NKoe5XOeduwbEQHU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkv9CrUACgkQForvXbEpPzQECQCcCGYQjHZ57QX5YEptEbbuom39 lf8Amwb9CelZEwMkx/1OP3bc9w2zRyp1 =IDm0 -----END PGP SIGNATURE----- --NKoe5XOeduwbEQHU-- From owner-freebsd-fs@FreeBSD.ORG Wed May 26 11:57:21 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98FBA1065675 for ; Wed, 26 May 2010 11:57:21 +0000 (UTC) (envelope-from stark@mapper.nl) Received: from smtp-out1.tiscali.nl (smtp-out1.tiscali.nl [195.241.79.176]) by mx1.freebsd.org (Postfix) with ESMTP id 561C08FC12 for ; Wed, 26 May 2010 11:57:21 +0000 (UTC) Received: from [82.170.17.27] (helo=mapper.nl) by smtp-out1.tiscali.nl with esmtp (Exim) (envelope-from ) id 1OHFEd-0002TV-U2; Wed, 26 May 2010 13:57:20 +0200 Received: from [10.58.235.50] by mapper.nl with esmtp (Exim 4.71 (FreeBSD)) (envelope-from ) id 1OHFEW-0007RZ-4R; Wed, 26 May 2010 13:57:12 +0200 Message-ID: <4BFD0C98.2010604@mapper.nl> Date: Wed, 26 May 2010 13:57:12 +0200 From: Mark Stapper User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Bartosz Stec References: <4BF94256.6080305@kkip.pl> In-Reply-To: <4BF94256.6080305@kkip.pl> X-Enigmail-Version: 1.0.1 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigF1D82001249816BB5FEC49D4" Cc: freebsd-fs@freebsd.org Subject: Re: Cannot import raidz pool after zero fill one hdd. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2010 11:57:21 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF1D82001249816BB5FEC49D4 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 23/05/2010 16:57, Bartosz Stec wrote: > After a while I've checked zpool status and ad0 was indeed offline, > but with about 200 errors? Shouldn't ZFS stop using this drive at all= ? ZFS just noticed it should have written/read from the drive, but couldn't/wouldn't because it's marked offline. > After reboot loader shows some LBA errors and that it cannot find > zpool. I've booted this machine using about month old freebsd-current > snapshot and got into fixit environment. > Here's output of zpool import: > http://img715.imageshack.us/img715/7860/19843139.jpg Seems like there's no label "disk1". Logically, because you zerofilled the disk... Did you re-label the partition? > > Zroot is in UNAVAIL state because of ad2p3 OFFLINE (which was never > offline), gpt/disk1 UNAVAIL, which is the same partition as ad1p3 > which is showed as ONLINE. Could it be ad2p3 got the label "disk0"? --------------enigF1D82001249816BB5FEC49D4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkv9DJoACgkQN9xNqOOVnWBFrQCdG2gkqZInI5052bKuWc1WyTBa ZEQAnRezTGwxIHnBqMYK14n9yVvK5XZY =vKvN -----END PGP SIGNATURE----- --------------enigF1D82001249816BB5FEC49D4-- From owner-freebsd-fs@FreeBSD.ORG Wed May 26 14:27:08 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3EBB1106566C for ; Wed, 26 May 2010 14:27:08 +0000 (UTC) (envelope-from shadow_user@rambler.ru) Received: from mxb.rambler.ru (mxb.rambler.ru [81.19.66.30]) by mx1.freebsd.org (Postfix) with ESMTP id EE3B78FC15 for ; Wed, 26 May 2010 14:27:07 +0000 (UTC) Received: from maild.rambler.ru (maild.rambler.ru [81.19.66.33]) by mxb.rambler.ru (Postfix) with ESMTP id A927B1BBF77 for ; Wed, 26 May 2010 18:09:52 +0400 (MSD) Received: from [188.186.250.247] (unknown [188.186.250.247]) (Authenticated sender: shadow_user@rambler.ru) by maild.rambler.ru (Postfix) with ESMTP id 809AA84414 for ; Wed, 26 May 2010 18:09:50 +0400 (MSD) Message-ID: <4BFD8E12.9040909@rambler.ru> Date: Wed, 26 May 2010 21:09:38 +0000 From: Dima Naumov User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100524 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: FS to physical disk block conversion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2010 14:27:08 -0000 I am sorry for my English. OK my problem: i do not understand how i can get real address of disk block, when i use macros fsbtodb() to convert fs address to physical disk block address i receive odd result, explain me please where my error. Example: struct fs *sblock; ino_t inode = 2; caddr_t inoblock; struct ufs2_dinode * ino; ... if(pread(dev_fd, ino, sizeof(struct ufs2_dinode),\ (sblock->fs_bsize * fsbtodb(sblock, \ ino_to_fsba(sblock, inode)))) != sizeof(struct ufs2_dinode)) error(__LINE__, errno); After executing this string a not get correct inode in "ino". Please explain, it important to me! From owner-freebsd-fs@FreeBSD.ORG Wed May 26 15:07:22 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BED53106566B for ; Wed, 26 May 2010 15:07:22 +0000 (UTC) (envelope-from gljennjohn@googlemail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4BA5F8FC1E for ; Wed, 26 May 2010 15:07:21 +0000 (UTC) Received: by fxm17 with SMTP id 17so1863420fxm.13 for ; Wed, 26 May 2010 08:07:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:in-reply-to:references:reply-to:x-mailer:mime-version :content-type:content-transfer-encoding; bh=HiBAYgLoSo86stkj/Oh5VcIl4YZC4F1BsjGG3EwfyME=; b=P5b2L7w9+Gc2PS2K4r+irK4glFCautF5pXhLlT9d6HX111wsGeEpUM78EQrlJI1ZZs xzJpmBK03mQt4nvD/GnA1hOvSyYe34HzWON/9vKuzerqkLcnZaJJRgJieTi9Yv77cXl7 7sLc0vN3SF5HsNbO8DqEH/0VwqRtOsFsDUXI8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :x-mailer:mime-version:content-type:content-transfer-encoding; b=hrHI2rTdgQ4FTx+i5fvKxTNp2n7HPYrkRZBf1VFBQhi/ttuhgDsaORFvTZTKD7zy8T 1xIX8IYcQZxGHSGqfL9T5CeCnq+UsfAEW2IFzYNjlnc2xhGSYCeMyE6L8ScmzFcQ8jAp Y9CIZFLIc4Ng4GJKbviA/mNQh6KspZ1Bqkhpw= Received: by 10.223.98.24 with SMTP id o24mr7749087fan.29.1274886440768; Wed, 26 May 2010 08:07:20 -0700 (PDT) Received: from ernst.jennejohn.org (p578E2F2C.dip.t-dialin.net [87.142.47.44]) by mx.google.com with ESMTPS id g10sm802160fai.12.2010.05.26.08.07.18 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 26 May 2010 08:07:18 -0700 (PDT) Date: Wed, 26 May 2010 17:07:16 +0200 From: Gary Jennejohn To: Dima Naumov Message-ID: <20100526170716.11f24c4a@ernst.jennejohn.org> In-Reply-To: <4BFD8E12.9040909@rambler.ru> References: <4BFD8E12.9040909@rambler.ru> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: FS to physical disk block conversion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gljennjohn@googlemail.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2010 15:07:22 -0000 On Wed, 26 May 2010 21:09:38 +0000 Dima Naumov wrote: > I am sorry for my English. OK my problem: i do not understand how i can > get real address of disk block, when i use macros fsbtodb() to convert > fs address to physical disk block address i receive odd result, explain > me please where my error. > Example: > struct fs *sblock; > ino_t inode = 2; > caddr_t inoblock; > struct ufs2_dinode * ino; > ... > if(pread(dev_fd, ino, sizeof(struct ufs2_dinode),\ > (sblock->fs_bsize * fsbtodb(sblock, \ > ino_to_fsba(sblock, inode)))) != sizeof(struct > ufs2_dinode)) > error(__LINE__, errno); > After executing this string a not get correct inode in "ino". Please > explain, it important to me! > See /usr/src/sbin/dump/traverse.c:getino() for how to do it. -- Gary Jennejohn From owner-freebsd-fs@FreeBSD.ORG Wed May 26 15:36:30 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E1481065670 for ; Wed, 26 May 2010 15:36:30 +0000 (UTC) (envelope-from rick@svn.kiwi-computer.com) Received: from svn.kiwi-computer.com (174-20-208-22.mpls.qwest.net [174.20.208.22]) by mx1.freebsd.org (Postfix) with SMTP id BC7DB8FC1D for ; Wed, 26 May 2010 15:36:29 +0000 (UTC) Received: (qmail 71128 invoked by uid 2000); 26 May 2010 15:09:46 -0000 Date: Wed, 26 May 2010 10:09:46 -0500 From: "Rick C. Petty" To: Dima Naumov Message-ID: <20100526150946.GA70983@kay.kiwi-computer.com> References: <4BFD8E12.9040909@rambler.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BFD8E12.9040909@rambler.ru> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@freebsd.org Subject: Re: FS to physical disk block conversion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd2009@kiwi-computer.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2010 15:36:30 -0000 On Wed, May 26, 2010 at 09:09:38PM +0000, Dima Naumov wrote: > I am sorry for my English. OK my problem: i do not understand how i can > get real address of disk block, when i use macros fsbtodb() to convert > fs address to physical disk block address i receive odd result, explain > me please where my error. > Example: > struct fs *sblock; > ino_t inode = 2; > caddr_t inoblock; > struct ufs2_dinode * ino; > ... > if(pread(dev_fd, ino, sizeof(struct ufs2_dinode),\ > (sblock->fs_bsize * fsbtodb(sblock, \ > ino_to_fsba(sblock, inode)))) != sizeof(struct > ufs2_dinode)) > error(__LINE__, errno); > After executing this string a not get correct inode in "ino". Please > explain, it important to me! Your calculation isn't even close to correct. First, ino_to_fsba() only gives the filesystem fragment number of the containing inode block; you need to add the offset to the actual inode. Second, the fsbtodb() macro is for computing the disk block number from the fragment number (disk blocks are typically 512 bytes), but unless you know the disk block size this macro is meaningless. Instead you should multiply by fs_fsize (it's better to shift by fs_fshift than to do the multiplication). Don't multiply by fs_bsize: the name "block" here is misleading since all UFS addresses are in multiples of fragment size. Another thing you may wish to consider (if you're concerned about performance) is reading at least one UFS block at a time and using pointer arithmetic to give you the dinode structs you need. Something like this makes more sense: uint8_t block_buffer[sblock->fs_bsize]; ... int64_t block_no = ino_to_fsba(sblock, inode); int64_t block_offset = block_no << sblock->fs_fshift; if (pread(dev_fd, block_buffer, sblock->fs_bsize, block_offset) != sblock->fs_bsize) ... struct ufs2_dinode * ino = (struct ufs2_dinode *)block_buffer + ino_to_fsbo(sblock, inode); HTH, -- Rick C. Petty From owner-freebsd-fs@FreeBSD.ORG Wed May 26 23:10:07 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35A36106564A for ; Wed, 26 May 2010 23:10:07 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (unknown [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0BE0E8FC14 for ; Wed, 26 May 2010 23:10:07 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o4QNA64s083011 for ; Wed, 26 May 2010 23:10:06 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o4QNA6BN083010; Wed, 26 May 2010 23:10:06 GMT (envelope-from gnats) Date: Wed, 26 May 2010 23:10:06 GMT Message-Id: <201005262310.o4QNA6BN083010@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Andriy Gapon Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Andriy Gapon List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2010 23:10:07 -0000 The following reply was made to PR bin/144214; it has been noted by GNATS. From: Andriy Gapon To: bug-followup@freebsd.org, c.kworr@gmail.com Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 Date: Thu, 27 May 2010 02:02:03 +0300 This is a multi-part message in MIME format. --------------070503050708060501010606 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Here's a new patch that, as I strongly believe, should fix the problem for real. I am sending "production ready" version of the patch, please keep "ZFS: gang block detected!" message in your sources during testing/verification. Thanks! -- Andriy Gapon --------------070503050708060501010606 Content-Type: text/plain; name="gang.diff" Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="gang.diff" ZGlmZiAtLWdpdCBhL3N5cy9ib290L3pmcy96ZnNpbXBsLmMgYi9zeXMvYm9vdC96ZnMvemZz aW1wbC5jCmluZGV4IDE0MDdlYjUuLjY0MDFjNmIgMTAwNjQ0Ci0tLSBhL3N5cy9ib290L3pm cy96ZnNpbXBsLmMKKysrIGIvc3lzL2Jvb3QvemZzL3pmc2ltcGwuYwpAQCAtOTU4LDEyICs5 NjksMTcgQEAgemlvX3JlYWRfZ2FuZyhzcGFfdCAqc3BhLCBjb25zdCBibGtwdHJfdCAqYnAs IGNvbnN0IGR2YV90ICpkdmEsIHZvaWQgKmJ1ZikKIAkJCWJyZWFrOwogCWlmICghdmRldiB8 fCAhdmRldi0+dl9yZWFkKQogCQlyZXR1cm4gKEVJTyk7Ci0JaWYgKHZkZXYtPnZfcmVhZCh2 ZGV2LCBicCwgJnppb19nYiwgb2Zmc2V0LCBTUEFfR0FOR0JMT0NLU0laRSkpCisJaWYgKHZk ZXYtPnZfcmVhZCh2ZGV2LCBOVUxMLCAmemlvX2diLCBvZmZzZXQsIFNQQV9HQU5HQkxPQ0tT SVpFKSkKIAkJcmV0dXJuIChFSU8pOwogCiAJZm9yIChpID0gMDsgaSA8IFNQQV9HQkhfTkJM S1BUUlM7IGkrKykgewotCQlpZiAoemlvX3JlYWQoc3BhLCAmemlvX2diLnpnX2Jsa3B0cltp XSwgYnVmKSkKKwkJYmxrcHRyX3QgKmdicCA9ICZ6aW9fZ2IuemdfYmxrcHRyW2ldOworCisJ CWlmIChCUF9JU19IT0xFKGdicCkpCisJCQljb250aW51ZTsKKwkJaWYgKHppb19yZWFkKHNw YSwgZ2JwLCBidWYpKQogCQkJcmV0dXJuIChFSU8pOworCQlidWYgPSAoY2hhciopYnVmICsg QlBfR0VUX1BTSVpFKGdicCk7CiAJfQogIAogCXJldHVybiAoMCk7CkBAIC05OTQsOSArMTAx MCw4IEBAIHppb19yZWFkKHNwYV90ICpzcGEsIGNvbnN0IGJsa3B0cl90ICpicCwgdm9pZCAq YnVmKQogCQkJY29udGludWU7CiAKIAkJaWYgKERWQV9HRVRfR0FORyhkdmEpKSB7Ci0JCQlw cmludGYoIlpGUzogZ2FuZyBibG9jayBkZXRlY3RlZCFcbiIpOwogCQkJaWYgKHppb19yZWFk X2dhbmcoc3BhLCBicCwgZHZhLCBidWYpKQotCQkJCXJldHVybiAoRUlPKTsgCisJCQkJY29u dGludWU7CiAJCX0gZWxzZSB7CiAJCQl2ZGV2aWQgPSBEVkFfR0VUX1ZERVYoZHZhKTsKIAkJ CW9mZnNldCA9IERWQV9HRVRfT0ZGU0VUKGR2YSk7Cg== --------------070503050708060501010606-- From owner-freebsd-fs@FreeBSD.ORG Thu May 27 08:35:39 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A450A106568A; Thu, 27 May 2010 08:35:39 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 57C5A8FC1E; Thu, 27 May 2010 08:35:37 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA13679; Thu, 27 May 2010 11:35:36 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OHYYx-000O7i-Vp; Thu, 27 May 2010 11:35:36 +0300 Message-ID: <4BFE2ED6.1070402@freebsd.org> Date: Thu, 27 May 2010 11:35:34 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100321) MIME-Version: 1.0 To: Robert Noland , freebsd-fs@freebsd.org References: <4BEBA334.6080101@icyb.net.ua> <4BEC040E.9080303@FreeBSD.org> In-Reply-To: <4BEC040E.9080303@FreeBSD.org> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Roman Divacky Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2010 08:35:39 -0000 I think I nailed this problem now. What was additionally needed was the following change: if (!vdev || !vdev->v_read) return (EIO); - if (vdev->v_read(vdev, bp, &zio_gb, offset, SPA_GANGBLOCKSIZE)) + if (vdev->v_read(vdev, NULL, &zio_gb, offset, SPA_GANGBLOCKSIZE)) return (EIO); Full patch is here: http://people.freebsd.org/~avg/boot-zfs-gang.diff Apparently I am not as smart as Roman :) because I couldn't find the bug by just starring at this rather small function (for couple of hours), so I had to reproduce the problem to catch it. Hence I am copying hackers@ to share couple of tricks that were new to me. Perhaps, they could help someone else some other day. First, after very helpful hints that I received in parallel from pjd and two Oracle/Sun developers it became very easy to reproduce a pool with files with gang blocks in them. One can set metaslab_gang_bang variable in metaslab.c to some value < 128K and then blocks with size greater than metaslab_gang_bang will be allocated as gang blocks with 25% chance. I personally did something similar but slightly more deterministic: --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c @@ -1572,6 +1572,12 @@ zio_dva_allocate(zio_t *zio) ASSERT3U(zio->io_prop.zp_ndvas, <=, spa_max_replication(spa)); ASSERT3U(zio->io_size, ==, BP_GET_PSIZE(bp)); + /*XXX XXX XXX XXX*/ + if (zio->io_size > 8 * 1024) { + return (zio_write_gang_block(zio)); + } + /*XXX XXX XXX XXX*/ + error = metaslab_alloc(spa, mc, zio->io_size, bp, zio->io_prop.zp_ndvas, zio->io_txg, NULL, 0); This ensured that any block > 8K would be a gang block. Then I compiled zfs.ko with this change and put it into a virtual machine where I created a pool and populated its root/boot filesystem with /boot directory. Booted in virtual machine from the new virtual disk and immediately hit the problem. So far, so good, but still no clue why zfsboot crashes upon encountering a gang block. So I decided to debug the crash with gdb. Standard steps: $ qemu ... -S -s $ gdb ... (gdb) target remote localhost:1234 Now I didn't want to single-step through the whole boot process, so I decided to get some help from gdb. Here's a trick: (gdb) add-symbol-file /usr/obj/usr/src/sys/boot/i386/gptzfsboot/gptzfsboot.out 0xa000 gptzfsboot.out is an ELF image produced by GCC, which then gets transformed into a raw binary and then into final BTX binary (gptzfsboot). gptzfsboot.out is built without much debugging data but at least it contains information about function names. Perhaps it's even possible to compile gptzfsboot.out with higher debug level, then debugging would be much more pleasant. 0xA000 is where _code_ from gptzfsboot.out ends up being loaded in memory. BTW, having only shallow knowledge about boot chain and BTX I didn't know this address. Another GDB trick helped me: (gdb) append memory boot.memdump 0x0 0x10000 This command dumps memory content in range 0x0-0x10000 to a file named boot.memdump. Then I produced a hex dump and searched for byte sequence with which gptzfsboot.bin starts (raw binary produced produced from gptzfsboot.out). Of course, memory dump should be taken after gptzfsboot is loaded into memory :) Catching the right moment requires a little bit of boot process knowledge. I caught it with: (gdb) b *0xC000 That is, memory dump was taken after gdb stopped at the above break point. After that it was a piece of cake. I set break point on zio_read_gang function (after add-symbol-file command) and the stepi-ed through the code (that is, instruction by instruction). The following command made it easier to see what's getting executed: (gdb) display/i 0xA000 + $eip I quickly stepped though the code and saw that a large value was passed to vdev_read as 'bytes' parameter. But this should have been 512. The oversized read into a buffer allocated on stack smashed the stack and that was the end. Backtracking the call chain in source code I immediately noticed the bp condition in vdev_read_phys and realized what the problem was. Hope this would be a useful reading. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Thu May 27 10:52:01 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98D24106566B for ; Thu, 27 May 2010 10:52:01 +0000 (UTC) (envelope-from admin@kkip.pl) Received: from mainframe.kkip.pl (kkip.pl [87.105.164.78]) by mx1.freebsd.org (Postfix) with ESMTP id 0E3728FC1B for ; Thu, 27 May 2010 10:52:00 +0000 (UTC) Received: from mb01.admin.lan.kkip.pl ([10.66.3.254]) by mainframe.kkip.pl with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.71 (FreeBSD)) (envelope-from ) id 1OHagm-000Eol-QA; Thu, 27 May 2010 12:51:57 +0200 Message-ID: <4BFE4EC9.2090102@kkip.pl> Date: Thu, 27 May 2010 12:51:53 +0200 From: Bartosz Stec User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.1.9) Gecko/20100317 Lightning/1.0b1 Thunderbird/3.0.4 MIME-Version: 1.0 To: Mark Stapper References: <4BF94256.6080305@kkip.pl> <4BFD0C98.2010604@mapper.nl> In-Reply-To: <4BFD0C98.2010604@mapper.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authenticated-User: admin@kkip.pl X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Spam-Score: -6.3 X-Spam-Score-Int: -62 X-Exim-Version: 4.71 (build at 02-Feb-2010 20:10:28) X-Date: 2010-05-27 12:51:57 X-Connected-IP: 10.66.3.254:1697 X-Message-Linecount: 61 X-Body-Linecount: 47 X-Message-Size: 2607 X-Body-Size: 1973 X-Received-Count: 1 X-Recipient-Count: 2 X-Local-Recipient-Count: 2 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 Cc: freebsd-fs@freebsd.org Subject: Re: Cannot import raidz pool after zero fill one hdd. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2010 10:52:01 -0000 W dniu 2010-05-26 13:57, Mark Stapper pisze: > On 23/05/2010 16:57, Bartosz Stec wrote: > >> After a while I've checked zpool status and ad0 was indeed offline, >> but with about 200 errors? Shouldn't ZFS stop using this drive at all? >> > ZFS just noticed it should have written/read from the drive, but > couldn't/wouldn't because it's marked offline. > >> After reboot loader shows some LBA errors and that it cannot find >> zpool. I've booted this machine using about month old freebsd-current >> snapshot and got into fixit environment. >> Here's output of zpool import: >> http://img715.imageshack.us/img715/7860/19843139.jpg >> > Seems like there's no label "disk1". Logically, because you zerofilled > the disk... > Did you re-label the partition? > >> Zroot is in UNAVAIL state because of ad2p3 OFFLINE (which was never >> offline), gpt/disk1 UNAVAIL, which is the same partition as ad1p3 >> which is showed as ONLINE. >> > Could it be ad2p3 got the label "disk0"? > > > Arrrrgh, I tried re-labelling and a lot of other things, without success. Now I am almost certain that It was my mistake - I didn't check if /dev/gpt/disk0 was placed on ad0 (yes, definitely stupid assumption), and probably it wasn't. It was probably ad2p3 which was labelled disk0. So It seems that I putted one partition offline, and zeroed other, while it was online, which ended with broken raidz1. ZFS seems to read metadata from partitions when you are trying to import zpools instead of rely on glabel, so changing labels ends only with zfs stop using them. Thankfully I have found hdd I used to temporarily store all data from this machine when I migrating my installation from mixed UFS/ZFS to ZFS-only and all data was still there. Dated march 31 but definitely better than nothing ;) Lesson taken, problem "solved". Still, I'm interested if it's possible to mark offline vdev online again before importing? Cheers. -- Bartosz Stec From owner-freebsd-fs@FreeBSD.ORG Thu May 27 11:09:57 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1CB0A106566C for ; Thu, 27 May 2010 11:09:57 +0000 (UTC) (envelope-from stark@mapper.nl) Received: from smtp-out2.tiscali.nl (smtp-out2.tiscali.nl [195.241.79.177]) by mx1.freebsd.org (Postfix) with ESMTP id CA2DB8FC21 for ; Thu, 27 May 2010 11:09:56 +0000 (UTC) Received: from [82.170.17.27] (helo=mapper.nl) by smtp-out2.tiscali.nl with esmtp (Exim) (envelope-from ) id 1OHayJ-0000SX-Ny; Thu, 27 May 2010 13:09:55 +0200 Received: from [10.58.235.50] by mapper.nl with esmtp (Exim 4.71 (FreeBSD)) (envelope-from ) id 1OHayC-000AuZ-Mx; Thu, 27 May 2010 13:09:48 +0200 Message-ID: <4BFE52F9.90208@mapper.nl> Date: Thu, 27 May 2010 13:09:45 +0200 From: Mark Stapper User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Bartosz Stec References: <4BF94256.6080305@kkip.pl> <4BFD0C98.2010604@mapper.nl> <4BFE4EC9.2090102@kkip.pl> In-Reply-To: <4BFE4EC9.2090102@kkip.pl> X-Enigmail-Version: 1.0.1 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig4B72E931008B35C328BAEE06" Cc: freebsd-fs@freebsd.org Subject: Re: Cannot import raidz pool after zero fill one hdd. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2010 11:09:57 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig4B72E931008B35C328BAEE06 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 27/05/2010 12:51, Bartosz Stec wrote: > Thankfully I have found hdd I used to temporarily store all data from > this machine when I migrating my installation from mixed UFS/ZFS to > ZFS-only and all data was still there. Dated march 31 but definitely > better than nothing ;) Lesson taken, problem "solved". yay backups! > > Still, I'm interested if it's possible to mark offline vdev online > again before importing? > Cheers. It's a moo point, if a pool was exported with an offline disk, and you cannot import, data is corrupt. --------------enig4B72E931008B35C328BAEE06 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkv+Uv8ACgkQN9xNqOOVnWBbTwCfRTz9Ca636hXYOUEkB0BFxom2 vrcAn0Eed3vKhZMhOQCKyq4yJRPvFAMm =AwvU -----END PGP SIGNATURE----- --------------enig4B72E931008B35C328BAEE06-- From owner-freebsd-fs@FreeBSD.ORG Thu May 27 14:35:19 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9574F106566C; Thu, 27 May 2010 14:35:19 +0000 (UTC) (envelope-from rnoland@FreeBSD.org) Received: from gizmo.2hip.net (gizmo.2hip.net [64.74.207.195]) by mx1.freebsd.org (Postfix) with ESMTP id 659018FC1A; Thu, 27 May 2010 14:35:19 +0000 (UTC) Received: from [10.170.20.44] (nat-170-142-177-44.tn.gov [170.142.177.44]) (authenticated bits=0) by gizmo.2hip.net (8.14.3/8.14.3) with ESMTP id o4REZGAd016324 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 May 2010 10:35:17 -0400 (EDT) (envelope-from rnoland@FreeBSD.org) Message-ID: <4BFE8306.6030006@FreeBSD.org> Date: Thu, 27 May 2010 09:34:46 -0500 From: Robert Noland Organization: FreeBSD User-Agent: Thunderbird 2.0.0.19 (X11/20090218) MIME-Version: 1.0 To: Andriy Gapon References: <4BEBA334.6080101@icyb.net.ua> <4BEC040E.9080303@FreeBSD.org> <4BFE2ED6.1070402@freebsd.org> In-Reply-To: <4BFE2ED6.1070402@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.6 required=5.0 tests=AWL, BAYES_00, FH_DATE_PAST_20XX, RDNS_DYNAMIC,SPF_SOFTFAIL autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on gizmo.2hip.net Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Roman Divacky Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2010 14:35:19 -0000 Andriy Gapon wrote: > > I think I nailed this problem now. > What was additionally needed was the following change: > if (!vdev || !vdev->v_read) > return (EIO); > - if (vdev->v_read(vdev, bp, &zio_gb, offset, SPA_GANGBLOCKSIZE)) > + if (vdev->v_read(vdev, NULL, &zio_gb, offset, SPA_GANGBLOCKSIZE)) > return (EIO); > > Full patch is here: > http://people.freebsd.org/~avg/boot-zfs-gang.diff > > Apparently I am not as smart as Roman :) because I couldn't find the bug by just > starring at this rather small function (for couple of hours), so I had to > reproduce the problem to catch it. Hence I am copying hackers@ to share couple > of tricks that were new to me. Perhaps, they could help someone else some other > day. Excellent, I'm glad that this is finally tested and seems to be working. When I initially added the code, I wasn't able to test it and it turned out the the issue that I was trying to resolve wasn't actually gang block related anyway. robert. > First, after very helpful hints that I received in parallel from pjd and two > Oracle/Sun developers it became very easy to reproduce a pool with files with > gang blocks in them. > One can set metaslab_gang_bang variable in metaslab.c to some value < 128K and > then blocks with size greater than metaslab_gang_bang will be allocated as gang > blocks with 25% chance. I personally did something similar but slightly more > deterministic: > --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c > +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c > @@ -1572,6 +1572,12 @@ zio_dva_allocate(zio_t *zio) > ASSERT3U(zio->io_prop.zp_ndvas, <=, spa_max_replication(spa)); > ASSERT3U(zio->io_size, ==, BP_GET_PSIZE(bp)); > > + /*XXX XXX XXX XXX*/ > + if (zio->io_size > 8 * 1024) { > + return (zio_write_gang_block(zio)); > + } > + /*XXX XXX XXX XXX*/ > + > error = metaslab_alloc(spa, mc, zio->io_size, bp, > zio->io_prop.zp_ndvas, zio->io_txg, NULL, 0); > > This ensured that any block > 8K would be a gang block. > Then I compiled zfs.ko with this change and put it into a virtual machine where > I created a pool and populated its root/boot filesystem with /boot directory. > Booted in virtual machine from the new virtual disk and immediately hit the problem. > > So far, so good, but still no clue why zfsboot crashes upon encountering a gang > block. > > So I decided to debug the crash with gdb. > Standard steps: > $ qemu ... -S -s > $ gdb > ... > (gdb) target remote localhost:1234 > > Now I didn't want to single-step through the whole boot process, so I decided to > get some help from gdb. Here's a trick: > (gdb) add-symbol-file /usr/obj/usr/src/sys/boot/i386/gptzfsboot/gptzfsboot.out > 0xa000 > > gptzfsboot.out is an ELF image produced by GCC, which then gets transformed into > a raw binary and then into final BTX binary (gptzfsboot). > gptzfsboot.out is built without much debugging data but at least it contains > information about function names. Perhaps it's even possible to compile > gptzfsboot.out with higher debug level, then debugging would be much more pleasant. > > 0xA000 is where _code_ from gptzfsboot.out ends up being loaded in memory. > BTW, having only shallow knowledge about boot chain and BTX I didn't know this > address. Another GDB trick helped me: > (gdb) append memory boot.memdump 0x0 0x10000 > > This command dumps memory content in range 0x0-0x10000 to a file named > boot.memdump. Then I produced a hex dump and searched for byte sequence with > which gptzfsboot.bin starts (raw binary produced produced from gptzfsboot.out). > > Of course, memory dump should be taken after gptzfsboot is loaded into memory :) > Catching the right moment requires a little bit of boot process knowledge. > I caught it with: > (gdb) b *0xC000 > > That is, memory dump was taken after gdb stopped at the above break point. > > After that it was a piece of cake. I set break point on zio_read_gang function > (after add-symbol-file command) and the stepi-ed through the code (that is, > instruction by instruction). The following command made it easier to see what's > getting executed: > (gdb) display/i 0xA000 + $eip > > I quickly stepped though the code and saw that a large value was passed to > vdev_read as 'bytes' parameter. But this should have been 512. The oversized > read into a buffer allocated on stack smashed the stack and that was the end. > > Backtracking the call chain in source code I immediately noticed the bp > condition in vdev_read_phys and realized what the problem was. > > Hope this would be a useful reading. From owner-freebsd-fs@FreeBSD.ORG Thu May 27 15:03:18 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 228C0106567C; Thu, 27 May 2010 15:03:18 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0AC928FC19; Thu, 27 May 2010 15:03:16 +0000 (UTC) Received: by fxm20 with SMTP id 20so95309fxm.13 for ; Thu, 27 May 2010 08:03:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.239.158.133 with SMTP id u5mr978444hbc.179.1274971207933; Thu, 27 May 2010 07:40:07 -0700 (PDT) Received: by 10.239.153.73 with HTTP; Thu, 27 May 2010 07:40:07 -0700 (PDT) In-Reply-To: <4BFE2ED6.1070402@freebsd.org> References: <4BEBA334.6080101@icyb.net.ua> <4BEC040E.9080303@FreeBSD.org> <4BFE2ED6.1070402@freebsd.org> Date: Thu, 27 May 2010 15:40:07 +0100 Message-ID: From: Doug Rabson To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Roman Divacky , Robert Noland Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2010 15:03:18 -0000 On 27 May 2010 09:35, Andriy Gapon wrote: > > > I think I nailed this problem now. > What was additionally needed was the following change: > if (!vdev || !vdev->v_read) > return (EIO); > - if (vdev->v_read(vdev, bp, &zio_gb, offset, SPA_GANGBLOCKSIZE)) > + if (vdev->v_read(vdev, NULL, &zio_gb, offset, SPA_GANGBLOCKSIZE)) > return (EIO); > > Full patch is here: > http://people.freebsd.org/~avg/boot-zfs-gang.diff > > Apparently I am not as smart as Roman :) because I couldn't find the bug by > just > starring at this rather small function (for couple of hours), so I had to > reproduce the problem to catch it. Hence I am copying hackers@ to share > couple > of tricks that were new to me. Perhaps, they could help someone else some > other > day. > > First, after very helpful hints that I received in parallel from pjd and > two > Oracle/Sun developers it became very easy to reproduce a pool with files > with > gang blocks in them. > One can set metaslab_gang_bang variable in metaslab.c to some value < 128K > and > then blocks with size greater than metaslab_gang_bang will be allocated as > gang > blocks with 25% chance. I personally did something similar but slightly > more > deterministic: > --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c > +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c > @@ -1572,6 +1572,12 @@ zio_dva_allocate(zio_t *zio) > ASSERT3U(zio->io_prop.zp_ndvas, <=, spa_max_replication(spa)); > ASSERT3U(zio->io_size, ==, BP_GET_PSIZE(bp)); > > + /*XXX XXX XXX XXX*/ > + if (zio->io_size > 8 * 1024) { > + return (zio_write_gang_block(zio)); > + } > + /*XXX XXX XXX XXX*/ > + > error = metaslab_alloc(spa, mc, zio->io_size, bp, > zio->io_prop.zp_ndvas, zio->io_txg, NULL, 0); > > This ensured that any block > 8K would be a gang block. > Then I compiled zfs.ko with this change and put it into a virtual machine > where > I created a pool and populated its root/boot filesystem with /boot > directory. > Booted in virtual machine from the new virtual disk and immediately hit the > problem. > > So far, so good, but still no clue why zfsboot crashes upon encountering a > gang > block. > > So I decided to debug the crash with gdb. > Standard steps: > $ qemu ... -S -s > $ gdb > ... > (gdb) target remote localhost:1234 > > Now I didn't want to single-step through the whole boot process, so I > decided to > get some help from gdb. Here's a trick: > (gdb) add-symbol-file > /usr/obj/usr/src/sys/boot/i386/gptzfsboot/gptzfsboot.out > 0xa000 > > gptzfsboot.out is an ELF image produced by GCC, which then gets transformed > into > a raw binary and then into final BTX binary (gptzfsboot). > gptzfsboot.out is built without much debugging data but at least it > contains > information about function names. Perhaps it's even possible to compile > gptzfsboot.out with higher debug level, then debugging would be much more > pleasant. > > 0xA000 is where _code_ from gptzfsboot.out ends up being loaded in memory. > BTW, having only shallow knowledge about boot chain and BTX I didn't know > this > address. Another GDB trick helped me: > (gdb) append memory boot.memdump 0x0 0x10000 > > This command dumps memory content in range 0x0-0x10000 to a file named > boot.memdump. Then I produced a hex dump and searched for byte sequence > with > which gptzfsboot.bin starts (raw binary produced produced from > gptzfsboot.out). > > Of course, memory dump should be taken after gptzfsboot is loaded into > memory :) > Catching the right moment requires a little bit of boot process knowledge. > I caught it with: > (gdb) b *0xC000 > > That is, memory dump was taken after gdb stopped at the above break point. > > After that it was a piece of cake. I set break point on zio_read_gang > function > (after add-symbol-file command) and the stepi-ed through the code (that is, > instruction by instruction). The following command made it easier to see > what's > getting executed: > (gdb) display/i 0xA000 + $eip > > I quickly stepped though the code and saw that a large value was passed to > vdev_read as 'bytes' parameter. But this should have been 512. The > oversized > read into a buffer allocated on stack smashed the stack and that was the > end. > > Backtracking the call chain in source code I immediately noticed the bp > condition in vdev_read_phys and realized what the problem was. > > Hope this would be a useful reading. > Excellent work - thanks for looking into this. I still think its easier to debug this code in userland using a shim that redirects the zfsboot i/o calls to simple read system calls... From owner-freebsd-fs@FreeBSD.ORG Thu May 27 15:13:21 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5226106564A; Thu, 27 May 2010 15:13:21 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 956BE8FC15; Thu, 27 May 2010 15:13:20 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA22282; Thu, 27 May 2010 18:13:12 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4BFE8C07.9010303@icyb.net.ua> Date: Thu, 27 May 2010 18:13:11 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100517) MIME-Version: 1.0 To: Doug Rabson References: <4BEBA334.6080101@icyb.net.ua> <4BEC040E.9080303@FreeBSD.org> <4BFE2ED6.1070402@freebsd.org> In-Reply-To: X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Robert Noland Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2010 15:13:21 -0000 on 27/05/2010 17:40 Doug Rabson said the following: > > Excellent work - thanks for looking into this. I still think its easier > to debug this code in userland using a shim that redirects the zfsboot > i/o calls to simple read system calls... Absolutely! That should much easier. Do you have such a shim that you could share? I'd be much obliged for it. And not only I, I think. Thanks! -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Thu May 27 21:31:59 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D27BB1065673 for ; Thu, 27 May 2010 21:31:59 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 8A5618FC27 for ; Thu, 27 May 2010 21:31:59 +0000 (UTC) Received: by vws12 with SMTP id 12so559214vws.13 for ; Thu, 27 May 2010 14:31:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=uZ5UXg0iGbzJdVtStS/isTqEQRPlkl450uiOzgEaz1E=; b=RQjbw8fQjUzpjTWBt/n0Y4svebzJN5Gv4vz8Y7j1o7euQ3hU9/5EgNV4jPLAwbsw4v sb9G5Xlm5meBsTH9/ZOxtGQZ4bPguctoYuM4xIRwAH+b6Em/Qwv3t5uO9SwTC0Rq2MO6 TaUpTlUST/VBQ6U1tlBrFJpkmpHnNX+TQfPmw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=f45/EDhjpNJOyQ4VUlZCBcglRwJIETyWGv3AlrJbe3A0DGTThlZXuLir7Wu9LY415e HcPBUTvISCIUHC4Q6JqqMRNJlOgDGHOaxEt5B5J+6KJ7d0VltP+1NLzp/CfocZfwZ3bB JZJn9T/RoMzYfmISnLa1r+0bgQzOl3syq+cl8= MIME-Version: 1.0 Received: by 10.229.88.147 with SMTP id a19mr2474455qcm.1.1274994252555; Thu, 27 May 2010 14:04:12 -0700 (PDT) Received: by 10.229.190.83 with HTTP; Thu, 27 May 2010 14:04:12 -0700 (PDT) Date: Thu, 27 May 2010 14:04:12 -0700 Message-ID: From: Garrett Cooper To: fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: gpart - how to point to the other disk? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2010 21:31:59 -0000 Hi FS folks, I'm currently looking at switching over bits of our existing infrastructure from boot0cfg / fdisk to gpart, and one of the things that not 100% clear is -- how do I get one disk to point to the other one? I can do this today with boot0cfg [...] -s 5, but AFAICT gpart bootcode should supersede this usage (or am I incorrect?). Thanks, -Garrett PS Please CC me on all messages as I'm not subscribed to fs@. From owner-freebsd-fs@FreeBSD.ORG Thu May 27 21:16:35 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E941106564A for ; Thu, 27 May 2010 21:16:35 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 578068FC19 for ; Thu, 27 May 2010 21:16:33 +0000 (UTC) Received: by vws12 with SMTP id 12so542793vws.13 for ; Thu, 27 May 2010 14:16:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=L1JJhZdnMBYH+39VEICD6tQQ7piMKhFPna1Bjw3w4aE=; b=T9bNisu+ldR7Qpy4lSopeVTlMQOZYafBzy1pF1jfMqqULt8pmv+VYduKBEVAdXr6NI kkSHU6XwKBOyF+UrB74NPKme3VDEvaVKpLdTP3yPijbqc/C2Y4lqVUUQ7oqmME1oB1iE JPtZGMji+ikaWO+KeXB7Hv5em6dGL6lKGfqLw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=e2Z1N47pnMphkPdlvKATiyszYB5Pb6XnXtcg2fc+QeadaNdwGB1gg1JuZ+t/6nO6wP 8xgw65lobakbHWgSkTvVkZNYaz/B8g5SHmlKCdo0K91p2urHeR4Ay8QU+3Q4ZwWpE3iG Tmb0OS5ujdu8/VLqWwvHnfUEsbsCRg8C9j4yA= MIME-Version: 1.0 Received: by 10.229.216.136 with SMTP id hi8mr2444668qcb.135.1274994986222; Thu, 27 May 2010 14:16:26 -0700 (PDT) Received: by 10.229.190.83 with HTTP; Thu, 27 May 2010 14:16:26 -0700 (PDT) In-Reply-To: References: Date: Thu, 27 May 2010 14:16:26 -0700 Message-ID: From: Garrett Cooper To: geom@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Thu, 27 May 2010 22:08:42 +0000 Cc: Subject: Re: gpart - how to point to the other disk? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2010 21:16:35 -0000 On Thu, May 27, 2010 at 2:04 PM, Garrett Cooper wrote: > Hi FS folks, > =A0 =A0I'm currently looking at switching over bits of our existing > infrastructure from boot0cfg / fdisk to gpart, and one of the things > that not 100% clear is -- how do I get one disk to point to the other > one? I can do this today with boot0cfg [...] -s 5, but AFAICT gpart > bootcode should supersede this usage (or am I incorrect?). > Thanks, > -Garrett > > PS Please CC me on all messages as I'm not subscribed to fs@. (fs@ BCCed) Sorry for the noise... I sent the request to the wrong list. Thanks, -Garrett From owner-freebsd-fs@FreeBSD.ORG Fri May 28 06:55:22 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8E341065676; Fri, 28 May 2010 06:55:22 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from freefall.freebsd.org (unknown [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AF9358FC15; Fri, 28 May 2010 06:55:22 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o4S6tMmb078963; Fri, 28 May 2010 06:55:22 GMT (envelope-from avg@freefall.freebsd.org) Received: (from avg@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o4S6tMGN078959; Fri, 28 May 2010 06:55:22 GMT (envelope-from avg) Date: Fri, 28 May 2010 06:55:22 GMT Message-Id: <201005280655.o4S6tMGN078959@freefall.freebsd.org> To: c.kworr@gmail.com, avg@FreeBSD.org, freebsd-fs@FreeBSD.org, avg@FreeBSD.org From: avg@FreeBSD.org Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 06:55:22 -0000 Synopsis: zfsboot fails on gang block after upgrade to zfs v14 State-Changed-From-To: open->analyzed State-Changed-By: avg State-Changed-When: Fri May 28 06:53:51 UTC 2010 State-Changed-Why: It seems that I've got interested and involved in this PR. Responsible-Changed-From-To: freebsd-fs->avg Responsible-Changed-By: avg Responsible-Changed-When: Fri May 28 06:53:51 UTC 2010 Responsible-Changed-Why: It seems that I've got interested and involved in this PR. http://www.freebsd.org/cgi/query-pr.cgi?pr=144214 From owner-freebsd-fs@FreeBSD.ORG Fri May 28 07:30:29 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A6E4106566B for ; Fri, 28 May 2010 07:30:29 +0000 (UTC) (envelope-from howard@leadmon.net) Received: from mail.leadmon.net (mail.leadmon.net [173.13.218.154]) by mx1.freebsd.org (Postfix) with ESMTP id 8E16C8FC08 for ; Fri, 28 May 2010 07:30:28 +0000 (UTC) Received: from HDLDESKTOP64 (hdl-desktop-64.leadmon.net [10.0.0.3]) (authenticated bits=0) by mail.leadmon.net (8.14.4/8.14.4/LNSG+SCOP+PSBL+LUBL+NJABL+SBL+DSBL+SORBS+CBL+RHSBL) with ESMTP id o4S7UQuj002311 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 28 May 2010 03:30:26 -0400 (EDT) (envelope-from howard@leadmon.net) X-DKIM: OpenDKIM Filter v2.0.1 mail.leadmon.net o4S7UQuj002311 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leadmon.net; s=default; t=1275031827; bh=2np9G7Dt3V2gAsM3WtOQr4hvAnehhMJ4d58Png0vRDs=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type; b=ZPTJ8JCS7fmqhUTdYFhEEGVEpi33mqDKH6R9gwOoa4ctUoT60CUMErELopSgNenru WMTz1OxArITe5gCi0XjL43Cu1Hm6PU/AKEO7PYIUiwpfilo1GRZeufO0TgUHYIXENO PpTuImYijOVduPbXfqRFX3J1WWgdIcEh01A7Y2m4= X-DomainKeys: Sendmail DomainKeys Filter v1.0.2 mail.leadmon.net o4S7UQuj002311 DomainKey-Signature: a=rsa-sha1; s=default; d=leadmon.net; c=simple; q=dns; h=x-senderid:authentication-results:from:to:subject:date: message-id:mime-version:content-type:x-mailer:thread-index: content-language:x-cr-puzzleid:x-cr-hashedpuzzle; b=VHXwhyluWbYh9vDKrzbeu+QnJAWFI2mVJ6qhVQX0uu/tw7Nbc2AODLnKMmdHPHdBq bwBxPtxrJAohdMu9OtGEdakkG4DYxgP51Xb3Yo/FnVnh9MXG2+RPRUQ5At35dPgp2Dx DdXC7B5JlG9h0MIgTWZQcNrbTdIJBQoAVdA7bUg= X-SenderID: Sendmail Sender-ID Filter v1.0.0 mail.leadmon.net o4S7UQuj002311 Authentication-Results: mail.leadmon.net; sender-id=pass header.from=howard@leadmon.net; auth=pass (LOGIN); spf=pass smtp.mfrom=howard@leadmon.net From: "Howard Leadmon" To: , Date: Fri, 28 May 2010 03:30:26 -0400 Message-ID: <060401cafe37$a411b240$ec3516c0$@net> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Acr+N6Hg1/YcQMbvSSSs4lYszqQsBA== Content-Language: en-us x-cr-puzzleid: {6C3A48FD-2C65-482D-A25C-1AB1B12D7675} x-cr-hashedpuzzle: mhg= BjbJ B/kd CJ3P DGhD EXU6 EYxW FmWe F5Yj GK3H JUg8 L8Am TeGe VFI0 VtrC WOmP; 2; YQBtAGQANgA0AEAAZgByAGUAZQBiAHMAZAAuAG8AcgBnADsAZgByAGUAZQBiAHMAZAAtAGYAcwBAAGYAcgBlAGUAYgBzAGQALgBvAHIAZwA=; Sosha1_v1; 7; {6C3A48FD-2C65-482D-A25C-1AB1B12D7675}; aABvAHcAYQByAGQAQABsAGUAYQBkAG0AbwBuAC4AbgBlAHQA; Fri, 28 May 2010 07:30:22 GMT; RgByAGUAZQBCAFMARAAgADgALgAxAC0AUAByAGUAcgBlAGwAZQBhAHMAZQAgAFAAYQBuAGkAYwAgAGEAbQBkADYANAAgAHcALwBaAEYAUwAuAC4A X-Virus-Scanned: clamav-milter 0.96.1 at vorlon.leadmon.net X-Virus-Status: Clean Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 07:30:29 -0000 I know there used to be some issues with this a while back, but thought with the 8.x FBSD servers most of this tuned itself, or then again maybe this is something different. For the first time I ever recall, I found my FreeBSD 8 server was crashed this past morning, with the following error: panic:kmem_malloc(131072):kmem_map to small: 1296826368 total allocated cupid=4 Not sure what other info might be useful, but I will include the dmesg boot info, as I know it shows memory and some tidbits on ZFS, if anything else would be useful to look at please let me know. Knock on wood it's been a long time since I have had any panics, so this one surprised me.. Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.1-PRERELEASE #9: Wed May 19 09:24:24 EDT 2010 howardl@vorlon.leadmon.net:/usr/obj/usr/src/sys/VORLON amd64 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2793.20-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0xf48 Family = f Model = 4 Stepping = 8 Features=0xbfebfbff Features2=0x649d AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 4117700608 (3926 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 2 core(s) x 2 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP/HT): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP/HT): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP/HT): APIC ID: 7 ioapic0: Changing APIC ID to 8 ioapic1: Changing APIC ID to 9 ioapic2: Changing APIC ID to 10 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 32-55 on motherboard ioapic2 irqs 64-87 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci1: on pcib1 pcib2: at device 0.0 on pci1 pci2: on pcib2 amr0: mem 0xf80f0000-0xf80fffff,0xfe9e0000-0xfe9fffff irq 46 at device 14.0 on pci2 amr0: Using 64-bit DMA amr0: [ITHREAD] amr0: delete logical drives supported by controller amr0: Firmware 521X, BIOS H430, 256MB RAM pcib3: at device 0.2 on pci1 pci3: on pcib3 pcib4: at device 4.0 on pci0 pci4: on pcib4 pcib5: at device 5.0 on pci0 pci5: on pcib5 pcib6: at device 0.0 on pci5 pci6: on pcib6 em0: port 0xecc0-0xecff mem 0xfe6e0000-0xfe6fffff irq 64 at device 7.0 on pci6 em0: [FILTER] em0: Ethernet address: 00:13:72:51:7e:43 pcib7: at device 0.2 on pci5 pci7: on pcib7 em1: port 0xdcc0-0xdcff mem 0xfe4e0000-0xfe4fffff irq 65 at device 8.0 on pci7 em1: [FILTER] em1: Ethernet address: 00:13:72:51:7e:44 pcib8: at device 6.0 on pci0 pci8: on pcib8 uhci0: port 0xbce0-0xbcff irq 16 at device 29.0 on pci0 uhci0: [ITHREAD] usbus0: on uhci0 uhci1: port 0xbcc0-0xbcdf irq 19 at device 29.1 on pci0 uhci1: [ITHREAD] usbus1: on uhci1 uhci2: port 0xbca0-0xbcbf irq 18 at device 29.2 on pci0 uhci2: [ITHREAD] usbus2: on uhci2 ehci0: mem 0xfeb00000-0xfeb003ff irq 23 at device 29.7 on pci0 ehci0: [ITHREAD] usbus3: EHCI version 1.0 usbus3: on ehci0 pcib9: at device 30.0 on pci0 pci9: on pcib9 vgapci0: port 0xcc00-0xccff mem 0xf0000000-0xf7ffffff,0xfe1f0000-0xfe1fffff irq 18 at device 13.0 on pci9 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] atrtc0: port 0x70-0x7f irq 8 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse, device ID 3 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff,0xec000-0xeffff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr e2800000e28 device_attach: est0 attach returned 6 p4tcc0: on cpu0 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr e2800000e28 device_attach: est1 attach returned 6 p4tcc1: on cpu1 est2: on cpu2 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr e2800000e28 device_attach: est2 attach returned 6 p4tcc2: on cpu2 est3: on cpu3 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr e2800000e28 device_attach: est3 attach returned 6 p4tcc3: on cpu3 est4: on cpu4 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr e2800000e28 device_attach: est4 attach returned 6 p4tcc4: on cpu4 est5: on cpu5 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr e2800000e28 device_attach: est5 attach returned 6 p4tcc5: on cpu5 est6: on cpu6 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr e2800000e28 device_attach: est6 attach returned 6 p4tcc6: on cpu6 est7: on cpu7 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr e2800000e28 device_attach: est7 attach returned 6 p4tcc7: on cpu7 ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present; to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS filesystem version 3 ZFS storage pool version 14 Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered acd0: CDROM at ata0-master UDMA33 amr0: delete logical drives supported by controller amrd0: on amr0 amrd0: 140160MB (287047680 sectors) RAID 1 (optimal) SMP: AP CPU #1 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #5 Launched! Root mount waiting for: usbus3 Root mount waiting for: usbus3 uhub3: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 ugen3.2: at usbus3 uhub4: on usbus3 uhub4: 2 ports with 2 removable, self powered Trying to mount root from zfs:tank/root em0: link state changed to UP --- Howard Leadmon From owner-freebsd-fs@FreeBSD.ORG Fri May 28 09:14:04 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E2851065672 for ; Fri, 28 May 2010 09:14:04 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 5061C8FC13 for ; Fri, 28 May 2010 09:14:02 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA08431; Fri, 28 May 2010 12:13:52 +0300 (EEST) (envelope-from avg@icyb.net.ua) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OHvdX-0002QE-RQ; Fri, 28 May 2010 12:13:51 +0300 Message-ID: <4BFF894F.4010008@icyb.net.ua> Date: Fri, 28 May 2010 12:13:51 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100321) MIME-Version: 1.0 To: Howard Leadmon References: <060401cafe37$a411b240$ec3516c0$__21811.5672738806$1275032797$gmane$org@net> In-Reply-To: <060401cafe37$a411b240$ec3516c0$__21811.5672738806$1275032797$gmane$org@net> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: amd64@freebsd.org, freebsd-fs@freebsd.org Subject: Re: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 09:14:04 -0000 on 28/05/2010 10:30 Howard Leadmon said the following: > I know there used to be some issues with this a while back, but thought > with the 8.x FBSD servers most of this tuned itself, or then again maybe > this is something different. > > > > For the first time I ever recall, I found my FreeBSD 8 server was crashed > this past morning, with the following error: > > > > panic:kmem_malloc(131072):kmem_map to small: 1296826368 total allocated > > cupid=4 O wow, this is an amd64 system (with 4G RAM) and you've got "kmem_map too small". That's very very strange. Are you sure you don't have any backwards-tuning in your configuration or environment? What value vm.kmem_size_max has? > Not sure what other info might be useful, but I will include the dmesg boot > info, as I know it shows memory and some tidbits on ZFS, if anything else > would be useful to look at please let me know. Knock on wood it's been a > long time since I have had any panics, so this one surprised me.. > > > > > > Copyright (c) 1992-2010 The FreeBSD Project. > > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > > The Regents of the University of California. All rights reserved. > > FreeBSD is a registered trademark of The FreeBSD Foundation. > > FreeBSD 8.1-PRERELEASE #9: Wed May 19 09:24:24 EDT 2010 > > howardl@vorlon.leadmon.net:/usr/obj/usr/src/sys/VORLON amd64 > > Timecounter "i8254" frequency 1193182 Hz quality 0 > > CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2793.20-MHz K8-class CPU) > > Origin = "GenuineIntel" Id = 0xf48 Family = f Model = 4 Stepping = 8 > > > Features=0xbfebfbff ,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > > Features2=0x649d > > AMD Features=0x20100800 > > AMD Features2=0x1 > > TSC: P-state invariant > > real memory = 4294967296 (4096 MB) > > avail memory = 4117700608 (3926 MB) > > ACPI APIC Table: > > FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs > > FreeBSD/SMP: 2 package(s) x 2 core(s) x 2 HTT threads > > cpu0 (BSP): APIC ID: 0 > > cpu1 (AP/HT): APIC ID: 1 > > cpu2 (AP): APIC ID: 2 > > cpu3 (AP/HT): APIC ID: 3 > > cpu4 (AP): APIC ID: 4 > > cpu5 (AP/HT): APIC ID: 5 > > cpu6 (AP): APIC ID: 6 > > cpu7 (AP/HT): APIC ID: 7 > > ioapic0: Changing APIC ID to 8 > > ioapic1: Changing APIC ID to 9 > > ioapic2: Changing APIC ID to 10 > > ioapic0 irqs 0-23 on motherboard > > ioapic1 irqs 32-55 on motherboard > > ioapic2 irqs 64-87 on motherboard > > kbd1 at kbdmux0 > > acpi0: on motherboard > > acpi0: [ITHREAD] > > acpi0: Power Button (fixed) > > Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 > > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 > > cpu0: on acpi0 > > cpu1: on acpi0 > > cpu2: on acpi0 > > cpu3: on acpi0 > > cpu4: on acpi0 > > cpu5: on acpi0 > > cpu6: on acpi0 > > cpu7: on acpi0 > > acpi_hpet0: iomem 0xfed00000-0xfed003ff on > acpi0 > > Timecounter "HPET" frequency 14318180 Hz quality 900 > > pcib0: port 0xcf8-0xcff on acpi0 > > pci0: on pcib0 > > pcib1: at device 2.0 on pci0 > > pci1: on pcib1 > > pcib2: at device 0.0 on pci1 > > pci2: on pcib2 > > amr0: mem > 0xf80f0000-0xf80fffff,0xfe9e0000-0xfe9fffff irq 46 at device 14.0 on pci2 > > amr0: Using 64-bit DMA > > amr0: [ITHREAD] > > amr0: delete logical drives supported by controller > > amr0: Firmware 521X, BIOS H430, 256MB RAM > > pcib3: at device 0.2 on pci1 > > pci3: on pcib3 > > pcib4: at device 4.0 on pci0 > > pci4: on pcib4 > > pcib5: at device 5.0 on pci0 > > pci5: on pcib5 > > pcib6: at device 0.0 on pci5 > > pci6: on pcib6 > > em0: port 0xecc0-0xecff > mem 0xfe6e0000-0xfe6fffff irq 64 at device 7.0 on pci6 > > em0: [FILTER] > > em0: Ethernet address: 00:13:72:51:7e:43 > > pcib7: at device 0.2 on pci5 > > pci7: on pcib7 > > em1: port 0xdcc0-0xdcff > mem 0xfe4e0000-0xfe4fffff irq 65 at device 8.0 on pci7 > > em1: [FILTER] > > em1: Ethernet address: 00:13:72:51:7e:44 > > pcib8: at device 6.0 on pci0 > > pci8: on pcib8 > > uhci0: port 0xbce0-0xbcff irq 16 > at device 29.0 on pci0 > > uhci0: [ITHREAD] > > usbus0: on uhci0 > > uhci1: port 0xbcc0-0xbcdf irq 19 > at device 29.1 on pci0 > > uhci1: [ITHREAD] > > usbus1: on uhci1 > > uhci2: port 0xbca0-0xbcbf irq 18 > at device 29.2 on pci0 > > uhci2: [ITHREAD] > > usbus2: on uhci2 > > ehci0: mem 0xfeb00000-0xfeb003ff > irq 23 at device 29.7 on pci0 > > ehci0: [ITHREAD] > > usbus3: EHCI version 1.0 > > usbus3: on ehci0 > > pcib9: at device 30.0 on pci0 > > pci9: on pcib9 > > vgapci0: port 0xcc00-0xccff mem > 0xf0000000-0xf7ffffff,0xfe1f0000-0xfe1fffff irq 18 at device 13.0 on pci9 > > isab0: at device 31.0 on pci0 > > isa0: on isab0 > > atapci0: port > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0 > > ata0: on atapci0 > > ata0: [ITHREAD] > > ata1: on atapci0 > > ata1: [ITHREAD] > > atrtc0: port 0x70-0x7f irq 8 on acpi0 > > fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 > > fdc0: [FILTER] > > atkbdc0: port 0x60,0x64 irq 1 on acpi0 > > atkbd0: irq 1 on atkbdc0 > > kbd0 at atkbd0 > > atkbd0: [GIANT-LOCKED] > > atkbd0: [ITHREAD] > > psm0: irq 12 on atkbdc0 > > psm0: [GIANT-LOCKED] > > psm0: [ITHREAD] > > psm0: model IntelliMouse, device ID 3 > > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 > > uart0: [FILTER] > > orm0: at iomem > 0xc0000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff,0xec000-0xeffff on isa0 > > sc0: at flags 0x100 on isa0 > > sc0: VGA <16 virtual consoles, flags=0x300> > > vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 > > ppc0: cannot reserve I/O port range > > est0: on cpu0 > > est: CPU supports Enhanced Speedstep, but is not recognized. > > est: cpu_vendor GenuineIntel, msr e2800000e28 > > device_attach: est0 attach returned 6 > > p4tcc0: on cpu0 > > est1: on cpu1 > > est: CPU supports Enhanced Speedstep, but is not recognized. > > est: cpu_vendor GenuineIntel, msr e2800000e28 > > device_attach: est1 attach returned 6 > > p4tcc1: on cpu1 > > est2: on cpu2 > > est: CPU supports Enhanced Speedstep, but is not recognized. > > est: cpu_vendor GenuineIntel, msr e2800000e28 > > device_attach: est2 attach returned 6 > > p4tcc2: on cpu2 > > est3: on cpu3 > > est: CPU supports Enhanced Speedstep, but is not recognized. > > est: cpu_vendor GenuineIntel, msr e2800000e28 > > device_attach: est3 attach returned 6 > > p4tcc3: on cpu3 > > est4: on cpu4 > > est: CPU supports Enhanced Speedstep, but is not recognized. > > est: cpu_vendor GenuineIntel, msr e2800000e28 > > device_attach: est4 attach returned 6 > > p4tcc4: on cpu4 > > est5: on cpu5 > > est: CPU supports Enhanced Speedstep, but is not recognized. > > est: cpu_vendor GenuineIntel, msr e2800000e28 > > device_attach: est5 attach returned 6 > > p4tcc5: on cpu5 > > est6: on cpu6 > > est: CPU supports Enhanced Speedstep, but is not recognized. > > est: cpu_vendor GenuineIntel, msr e2800000e28 > > device_attach: est6 attach returned 6 > > p4tcc6: on cpu6 > > est7: on cpu7 > > est: CPU supports Enhanced Speedstep, but is not recognized. > > est: cpu_vendor GenuineIntel, msr e2800000e28 > > device_attach: est7 attach returned 6 > > p4tcc7: on cpu7 > > ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is > present; > > to enable, add "vfs.zfs.prefetch_disable=0" to > /boot/loader.conf. > > ZFS filesystem version 3 > > ZFS storage pool version 14 > > Timecounters tick every 1.000 msec > > usbus0: 12Mbps Full Speed USB v1.0 > > usbus1: 12Mbps Full Speed USB v1.0 > > usbus2: 12Mbps Full Speed USB v1.0 > > usbus3: 480Mbps High Speed USB v2.0 > > ugen0.1: at usbus0 > > uhub0: on usbus0 > > ugen1.1: at usbus1 > > uhub1: on usbus1 > > ugen2.1: at usbus2 > > uhub2: on usbus2 > > ugen3.1: at usbus3 > > uhub3: on usbus3 > > uhub0: 2 ports with 2 removable, self powered > > uhub1: 2 ports with 2 removable, self powered > > uhub2: 2 ports with 2 removable, self powered > > acd0: CDROM at ata0-master UDMA33 > > amr0: delete logical drives supported by controller > > amrd0: on amr0 > > amrd0: 140160MB (287047680 sectors) RAID 1 (optimal) > > SMP: AP CPU #1 Launched! > > SMP: AP CPU #6 Launched! > > SMP: AP CPU #3 Launched! > > SMP: AP CPU #4 Launched! > > SMP: AP CPU #7 Launched! > > SMP: AP CPU #2 Launched! > > SMP: AP CPU #5 Launched! > > Root mount waiting for: usbus3 > > Root mount waiting for: usbus3 > > uhub3: 6 ports with 6 removable, self powered > > Root mount waiting for: usbus3 > > ugen3.2: at usbus3 > > uhub4: on > usbus3 > > uhub4: 2 ports with 2 removable, self powered > > Trying to mount root from zfs:tank/root > > em0: link state changed to UP > > > > > > --- > > Howard Leadmon > > > -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Fri May 28 13:32:30 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C3721065676; Fri, 28 May 2010 13:32:30 +0000 (UTC) (envelope-from mcj@bluetonic.org) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id D1E4C8FC18; Fri, 28 May 2010 13:32:29 +0000 (UTC) Received: by pwj4 with SMTP id 4so673707pwj.13 for ; Fri, 28 May 2010 06:32:29 -0700 (PDT) Received: by 10.140.179.25 with SMTP id b25mr204612rvf.54.1275052023691; Fri, 28 May 2010 06:07:03 -0700 (PDT) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx.google.com with ESMTPS id h11sm2035255rvm.9.2010.05.28.06.07.02 (version=SSLv3 cipher=RC4-MD5); Fri, 28 May 2010 06:07:02 -0700 (PDT) Received: by pvg16 with SMTP id 16so542527pvg.13 for ; Fri, 28 May 2010 06:07:01 -0700 (PDT) Received: by 10.143.85.4 with SMTP id n4mr130177wfl.312.1275052021203; Fri, 28 May 2010 06:07:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.143.30.7 with HTTP; Fri, 28 May 2010 06:06:41 -0700 (PDT) In-Reply-To: <4BFF894F.4010008@icyb.net.ua> References: <060401cafe37$a411b240$ec3516c0$__21811.5672738806$1275032797$gmane$org@net> <4BFF894F.4010008@icyb.net.ua> From: Carey Jones Date: Fri, 28 May 2010 08:06:41 -0500 Message-ID: To: amd64@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 13:32:30 -0000 On Fri, May 28, 2010 at 4:13 AM, Andriy Gapon wrote: > on 28/05/2010 10:30 Howard Leadmon said the following: > > panic:kmem_malloc(131072):kmem_map to small: 1296826368 total allocated > > > > cupid=4 > > O wow, this is an amd64 system (with 4G RAM) and you've got "kmem_map too > small". That's very very strange. > Are you sure you don't have any backwards-tuning in your configuration or > environment? > What value vm.kmem_size_max has? > I had a similar issue this morning, while doing a large compile (qt4). I do not have a debug kernel going, so I can't get a dump or anything, but I had exactly the same error as above (a different total allocated, of course, but in the same neighborhood). If this happens again, I'll get a debug kernel compiled, but hopefully it doesn't. :-) FreeBSD 8.1-PRERELEASE #0: Wed May 19 00:23:09 CDT 2010 root@ark.bluetonc.org:/usr/obj/usr/src/sys/ARK amd64 real memory = 4294967296 (4096 MB) avail memory = 3852931072 (3674 MB) mcj@ark ~ % sysctl vm.kmem_size_max vm.kmem_size_max: 329853485875 Not a lot of info here, I know, but it's just another data point. Thanks, -c From owner-freebsd-fs@FreeBSD.ORG Fri May 28 13:36:41 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B0E41065676 for ; Fri, 28 May 2010 13:36:41 +0000 (UTC) (envelope-from james-freebsd-fs2@jrv.org) Received: from mail.jrv.org (adsl-70-243-84-13.dsl.austtx.swbell.net [70.243.84.13]) by mx1.freebsd.org (Postfix) with ESMTP id A875B8FC17 for ; Fri, 28 May 2010 13:36:40 +0000 (UTC) Received: from kremvax.housenet.jrv (kremvax.housenet.jrv [192.168.3.124]) by mail.jrv.org (8.14.3/8.14.3) with ESMTP id o4SDacFZ032172 for ; Fri, 28 May 2010 08:36:38 -0500 (CDT) (envelope-from james-freebsd-fs2@jrv.org) Authentication-Results: mail.jrv.org; domainkeys=pass (testing) header.from=james-freebsd-fs2@jrv.org DomainKey-Signature: a=rsa-sha1; s=enigma; d=jrv.org; c=nofws; q=dns; h=message-id:date:from:user-agent:mime-version:to:subject: content-type:content-transfer-encoding; b=nxB3qL9MwOZ9dSAMRzkvxQZj9FrOu8J4QVbGqArRddaKPD1Fj44IEMAV/1i5zv0on TDZhmO/ySczB++OTNcKJRq5dT+QXUICm0KPtoTVUPnL18Rm3LgjajwKuyK0TCHFD9Oa nA1nA20ZHSj9vcUI7eDsKvGobm2tiLdW1UE3tJ0= Message-ID: <4BFFC6E6.2070505@jrv.org> Date: Fri, 28 May 2010 08:36:38 -0500 From: "James R. Van Artsdalen" User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: freebsd-fs Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: ZFS: how to replace a dead disk? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 13:36:41 -0000 What's the right way to replace a dead disk under ZFS? When ada1 failed I tried "zpool replace jwrc ada1" and when it finished I got this: root@cyclone ~]# zpool status pool: jwrc state: DEGRADED status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scrub: resilver completed after 14h55m with 11 errors on Fri May 28 04:28:56 2010 config: NAME STATE READ WRITE CKSUM jwrc DEGRADED 0 0 11 raidz1 DEGRADED 0 0 23 ada2 ONLINE 0 0 0 889M resilvered replacing DEGRADED 0 0 0 ada1/old UNAVAIL 0 256K 0 cannot open ada1 ONLINE 0 0 0 1.47T resilvered ada3 ONLINE 0 0 0 879M resilvered ada4 ONLINE 0 0 0 808M resilvered errors: 5 data errors, use '-v' for a list --- It says "replacing" and that the device, vdev and pool are degraded, yet the "resilver" finished hours ago. I cannot detach the ada1/old entry. Is there some other command I should have used to remove the dead ada1 device? This kernel 206111, roughly April 1, on amd64 From owner-freebsd-fs@FreeBSD.ORG Fri May 28 13:45:53 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 545951065678 for ; Fri, 28 May 2010 13:45:53 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta02.westchester.pa.mail.comcast.net (qmta02.westchester.pa.mail.comcast.net [76.96.62.24]) by mx1.freebsd.org (Postfix) with ESMTP id 02A588FC13 for ; Fri, 28 May 2010 13:45:52 +0000 (UTC) Received: from omta03.westchester.pa.mail.comcast.net ([76.96.62.27]) by qmta02.westchester.pa.mail.comcast.net with comcast id NyJz1e0010bG4ec521lt4K; Fri, 28 May 2010 13:45:53 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta03.westchester.pa.mail.comcast.net with comcast id P1lq1e00N3S48mS3P1lrSc; Fri, 28 May 2010 13:45:52 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 7D1229B418; Fri, 28 May 2010 06:45:49 -0700 (PDT) Date: Fri, 28 May 2010 06:45:49 -0700 From: Jeremy Chadwick To: Andriy Gapon Message-ID: <20100528134549.GA75411@icarus.home.lan> References: <060401cafe37$a411b240$ec3516c0$@net> <4BFF894F.4010008@icyb.net.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BFF894F.4010008@icyb.net.ua> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: amd64@freebsd.org, freebsd-fs@freebsd.org Subject: Re: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 13:45:53 -0000 On Fri, May 28, 2010 at 12:13:51PM +0300, Andriy Gapon wrote: > on 28/05/2010 10:30 Howard Leadmon said the following: > > I know there used to be some issues with this a while back, but thought > > with the 8.x FBSD servers most of this tuned itself, or then again maybe > > this is something different. > > > > > > > > For the first time I ever recall, I found my FreeBSD 8 server was crashed > > this past morning, with the following error: > > > > > > > > panic:kmem_malloc(131072):kmem_map to small: 1296826368 total allocated > > > > cupid=4 > > O wow, this is an amd64 system (with 4G RAM) and you've got "kmem_map too > small". That's very very strange. It is? On amd64, vm.kmem_size (not vm.kmem_size_max) is what has to be increased. I can point folks to the "official" statement from pjd@ and some others if need be. For a very long time I questioned this because for an even longer amount of time we were being told to increase vm.kmem_size_max. vm.kmem_size_max, by default, is already huge on amd64 (~320GB or something like that). Proof: vm.kmem_size_max: 329853485875 To the OP: you will need to increase vm.kmem_size in /boot/loader.conf and reboot the system. "What value do I pick?" With 4GB, I would recommend you use these two variables: vm.kmem_size="2048M" vfs.zfs.arc_max="1536M" This will increase the available kmem, and also limit the ARC size explicitly to nothing larger than 1.5GB. This should stabilise your system. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Fri May 28 13:49:29 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DFAC2106564A for ; Fri, 28 May 2010 13:49:29 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta03.westchester.pa.mail.comcast.net (qmta03.westchester.pa.mail.comcast.net [76.96.62.32]) by mx1.freebsd.org (Postfix) with ESMTP id 8A95A8FC0A for ; Fri, 28 May 2010 13:49:29 +0000 (UTC) Received: from omta24.westchester.pa.mail.comcast.net ([76.96.62.76]) by qmta03.westchester.pa.mail.comcast.net with comcast id NzeE1e0091ei1Bg531pVii; Fri, 28 May 2010 13:49:29 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta24.westchester.pa.mail.comcast.net with comcast id P1pU1e0013S48mS3k1pUi5; Fri, 28 May 2010 13:49:29 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id E45589B418; Fri, 28 May 2010 06:49:26 -0700 (PDT) Date: Fri, 28 May 2010 06:49:26 -0700 From: Jeremy Chadwick To: "James R. Van Artsdalen" Message-ID: <20100528134926.GB75411@icarus.home.lan> References: <4BFFC6E6.2070505@jrv.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BFFC6E6.2070505@jrv.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs Subject: Re: ZFS: how to replace a dead disk? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 13:49:30 -0000 On Fri, May 28, 2010 at 08:36:38AM -0500, James R. Van Artsdalen wrote: > What's the right way to replace a dead disk under ZFS? > > When ada1 failed I tried "zpool replace jwrc ada1" and when it finished > I got this: > > root@cyclone ~]# zpool status > pool: jwrc > state: DEGRADED > status: One or more devices has experienced an error resulting in data > corruption. Applications may be affected. > action: Restore the file in question if possible. Otherwise restore the > entire pool from backup. > see: http://www.sun.com/msg/ZFS-8000-8A > scrub: resilver completed after 14h55m with 11 errors on Fri May 28 > 04:28:56 2010 > config: > > NAME STATE READ WRITE CKSUM > jwrc DEGRADED 0 0 11 > raidz1 DEGRADED 0 0 23 > ada2 ONLINE 0 0 0 889M resilvered > replacing DEGRADED 0 0 0 > ada1/old UNAVAIL 0 256K 0 cannot open > ada1 ONLINE 0 0 0 1.47T resilvered > ada3 ONLINE 0 0 0 879M resilvered > ada4 ONLINE 0 0 0 808M resilvered > > errors: 5 data errors, use '-v' for a list > > --- > > It says "replacing" and that the device, vdev and pool are degraded, yet > the "resilver" finished hours ago. I cannot detach the ada1/old entry. > > Is there some other command I should have used to remove the dead ada1 > device? What version of FreeBSD? Please provide uname -a output and not "8.0" or something equally as terse. Some clarification: you didn't remove the device, you simply told ZFS to assuming that the device had been replaced. What did you do (both physically and software/command-line-wise) *prior* to issuing "zpool replace jwrc ada1"? -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Fri May 28 14:09:33 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2C0E106564A; Fri, 28 May 2010 14:09:33 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (unknown [IPv6:2001:470:1f09:679::1]) by mx1.freebsd.org (Postfix) with ESMTP id A744B8FC16; Fri, 28 May 2010 14:09:33 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id 90DBCC400F; Fri, 28 May 2010 14:09:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on muon.cran.org.uk X-Spam-Level: X-Spam-Status: No, score=-2.7 required=8.0 tests=AWL,BAYES_00,RDNS_DYNAMIC autolearn=no version=3.2.5 Received: from [192.168.1.140] (87-194-158-129.bethere.co.uk [87.194.158.129]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA; Fri, 28 May 2010 14:09:32 +0000 (UTC) Message-ID: <4BFFCE8F.9090409@cran.org.uk> Date: Fri, 28 May 2010 15:09:19 +0100 From: Bruce Cran User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Jeremy Chadwick References: <060401cafe37$a411b240$ec3516c0$@net> <4BFF894F.4010008@icyb.net.ua> <20100528134549.GA75411@icarus.home.lan> In-Reply-To: <20100528134549.GA75411@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: amd64@freebsd.org, freebsd-fs@freebsd.org, Andriy Gapon Subject: Re: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 14:09:34 -0000 On 28/05/2010 14:45, Jeremy Chadwick wrote: > It is? On amd64, vm.kmem_size (not vm.kmem_size_max) is what has to be > increased. I can point folks to the "official" statement from pjd@ and > some others if need be. For a very long time I questioned this because > for an even longer amount of time we were being told to increase > vm.kmem_size_max. > It seems the wiki should probably be updated - it still says 8.0 and 9-CURRENT shouldn't require tuning - http://wiki.freebsd.org/ZFSTuningGuide says: "FreeBSD 7.2+ has improved kernel memory allocation strategy and no tuning may be necessary on systems with more than 2 GB of RAM." -- Bruce Cran From owner-freebsd-fs@FreeBSD.ORG Fri May 28 14:49:49 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78E331065673; Fri, 28 May 2010 14:49:49 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9C52C8FC1F; Fri, 28 May 2010 14:49:48 +0000 (UTC) Received: by vws12 with SMTP id 12so1322673vws.13 for ; Fri, 28 May 2010 07:49:47 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.63.68 with SMTP id a4mr307723vci.9.1275058187678; Fri, 28 May 2010 07:49:47 -0700 (PDT) Received: by 10.220.199.134 with HTTP; Fri, 28 May 2010 07:49:47 -0700 (PDT) In-Reply-To: <4BFE8C07.9010303@icyb.net.ua> References: <4BEBA334.6080101@icyb.net.ua> <4BEC040E.9080303@FreeBSD.org> <4BFE2ED6.1070402@freebsd.org> <4BFE8C07.9010303@icyb.net.ua> Date: Fri, 28 May 2010 15:49:47 +0100 Message-ID: From: Doug Rabson To: Andriy Gapon Content-Type: multipart/mixed; boundary=e0cb4e8879015834f40487a89e4e X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Robert Noland Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 14:49:49 -0000 --e0cb4e8879015834f40487a89e4e Content-Type: text/plain; charset=ISO-8859-1 On 27 May 2010 16:13, Andriy Gapon wrote: > on 27/05/2010 17:40 Doug Rabson said the following: > > > > Excellent work - thanks for looking into this. I still think its easier > > to debug this code in userland using a shim that redirects the zfsboot > > i/o calls to simple read system calls... > > Absolutely! That should much easier. > Do you have such a shim that you could share? > I'd be much obliged for it. And not only I, I think. > Thanks! > Attached. I thought I sent it to the list before but perhaps I only sent to one of the participants in the last gang block thread. --e0cb4e8879015834f40487a89e4e Content-Type: application/octet-stream; name="zfstest.c" Content-Disposition: attachment; filename="zfstest.c" Content-Transfer-Encoding: base64 X-Attachment-Id: f_g9r4pi2m0 I2luY2x1ZGUgPHN5cy9wYXJhbS5oPgojaW5jbHVkZSA8c3lzL3F1ZXVlLmg+CiNpbmNsdWRlIDxm Y250bC5oPgojaW5jbHVkZSA8c3RkaW50Lmg+CiNpbmNsdWRlIDxzdGRpby5oPgojaW5jbHVkZSA8 c3RyaW5nLmg+CiNpbmNsdWRlIDxzdGRhcmcuaD4KI2luY2x1ZGUgPHN0ZGRlZi5oPgojaW5jbHVk ZSA8c3RkbGliLmg+CiNpbmNsdWRlIDxlcnJuby5oPgoKI2RlZmluZSBOQkJZIDgKCnZvaWQKcGFn ZXJfb3V0cHV0KGNvbnN0IGNoYXIgKmxpbmUpCnsKCXByaW50ZigiJXMiLCBsaW5lKTsKfQoKI2lu Y2x1ZGUgInpmc2ltcGwuYyIKCnN0YXRpYyBpbnQKdmRldl9yZWFkKHZkZXZfdCAqdmRldiwgdm9p ZCAqcHJpdiwgb2ZmX3Qgb2ZmLCB2b2lkICpidWYsIHNpemVfdCBieXRlcykKewoJaW50IGZkID0g KihpbnQgKikgcHJpdjsKCglpZiAocHJlYWQoZmQsIGJ1ZiwgYnl0ZXMsIG9mZikgIT0gYnl0ZXMp CgkJcmV0dXJuIC0xOwoJcmV0dXJuIDA7Cn0KCnN0YXRpYyBpbnQKemZzX3JlYWQoc3BhX3QgKnNw YSwgZG5vZGVfcGh5c190ICpkbiwgdm9pZCAqYnVmLCBzaXplX3Qgc2l6ZSwgb2ZmX3Qgb2ZmKQp7 Cgljb25zdCB6bm9kZV9waHlzX3QgKnpwID0gKGNvbnN0IHpub2RlX3BoeXNfdCAqKSBkbi0+ZG5f Ym9udXM7CglzaXplX3QgbjsKCWludCByYzsKCgluID0gc2l6ZTsKCWlmIChvZmYgKyBuID4genAt PnpwX3NpemUpCgkJbiA9IHpwLT56cF9zaXplIC0gb2ZmOwoJCglyYyA9IGRub2RlX3JlYWQoc3Bh LCBkbiwgb2ZmLCBidWYsIG4pOwoJaWYgKHJjKQoJCXJldHVybiAocmMpOwoKCXJldHVybiAobik7 Cn0KCmludAptYWluKGludCBhcmdjLCBjaGFyKiogYXJndikKewoJaW50IGksIG4sIG9mZjsKCWlu dCBmZFs5OV07CglzcGFfdCAqc3BhOwoJZG5vZGVfcGh5c190IGRuOwoJY2hhciBidWZbNTEyXTsK Cgl6ZnNfaW5pdCgpOwoJaWYgKGFyZ2MgPT0gMSkgewoJCXN0YXRpYyBjaGFyICphdltdID0gewoJ CQkiemZzdGVzdCIsICIvZGV2L2RhMHAyIiwgIi9kZXYvZGExcDIiLCAiL2Rldi9kYTJwMiIsCgkJ CU5VTEwsCgkJfTsKCQlhcmdjID0gNDsKCQlhcmd2ID0gYXY7Cgl9Cglmb3IgKGkgPSAxOyBpIDwg YXJnYzsgaSsrKSB7CgkJZmRbaV0gPSBvcGVuKGFyZ3ZbaV0sIE9fUkRPTkxZKTsKCQlpZiAoZmRb aV0gPCAwKQoJCQljb250aW51ZTsKCQlpZiAodmRldl9wcm9iZSh2ZGV2X3JlYWQsICZmZFtpXSwg TlVMTCkgIT0gMCkKCQkJY2xvc2UoZmRbaV0pOwoJfQoJc3BhX2FsbF9zdGF0dXMoKTsKCglzcGEg PSBTVEFJTFFfRklSU1QoJnpmc19wb29scyk7CglpZiAoIXNwYSB8fCB6ZnNfbW91bnRfcG9vbChz cGEpKQoJCWV4aXQoMSk7CgoJaWYgKHpmc19sb29rdXAoc3BhLCAiemZzLmMiLCAmZG4pKQoJCWV4 aXQoMSk7CgoJb2ZmID0gMDsKCWRvIHsKCQluID0gemZzX3JlYWQoc3BhLCAmZG4sIGJ1ZiwgNTEy LCBvZmYpOwoJCXdyaXRlKDEsIGJ1Ziwgbik7CgkJb2ZmICs9IG47Cgl9IHdoaWxlIChuID09IDUx Mik7Cn0K --e0cb4e8879015834f40487a89e4e-- From owner-freebsd-fs@FreeBSD.ORG Fri May 28 14:52:01 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 969CA1065677; Fri, 28 May 2010 14:52:01 +0000 (UTC) (envelope-from howard@leadmon.net) Received: from mail.leadmon.net (mail.leadmon.net [173.13.218.154]) by mx1.freebsd.org (Postfix) with ESMTP id E05AA8FC24; Fri, 28 May 2010 14:52:00 +0000 (UTC) Received: from HDLDESKTOP64 (hdl-desktop-64.leadmon.net [10.0.0.3]) (authenticated bits=0) by mail.leadmon.net (8.14.4/8.14.4/LNSG+SCOP+PSBL+LUBL+NJABL+SBL+DSBL+SORBS+CBL+RHSBL) with ESMTP id o4SEpxUE002645 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 28 May 2010 10:51:59 -0400 (EDT) (envelope-from howard@leadmon.net) X-DKIM: OpenDKIM Filter v2.0.1 mail.leadmon.net o4SEpxUE002645 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leadmon.net; s=default; t=1275058319; bh=9WExckdNJqQDgTS2N2MbFN0SCHUtcpPzlUBu9J71Ha8=; h=From:To:Cc:References:In-Reply-To:Subject:Date:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding; b=HhjeHJICXY8gSyejutBpFFdIgk/0SeCv8Xbfa3RrLwf6QuQ3ywVZezKN1NVObvHcY iPBtTiI1hpm7cnMJJWXXu8sIet2Gfz8x5Q5YxlkM740PiLYeudZ7eGDEpgfupiHoJq CTGr5ojVL73K+RcxNvKTMlJiVQxFY7/UYDsLI3rw= X-DomainKeys: Sendmail DomainKeys Filter v1.0.2 mail.leadmon.net o4SEpxUE002645 DomainKey-Signature: a=rsa-sha1; s=default; d=leadmon.net; c=simple; q=dns; h=x-senderid:authentication-results:from:to:cc:references: in-reply-to:subject:date:message-id:mime-version:content-type: content-transfer-encoding:x-mailer:thread-index:content-language; b=AqvFviPoZCqcU2DRDhEJ0+fg+1NEpDWfVDHQ3m7h8o/VtvPRf52UjlaOIobM5g1Uu Z66ZOXqjOh1uHBfH67I6psQZqKIg1sx82rwWJ5GImjUuIeMZVH/86FFY8A5FxPS7Yla 1I1oSesuB85fPD4tzEgeLDBQCenGBt7tJM41tq8= X-SenderID: Sendmail Sender-ID Filter v1.0.0 mail.leadmon.net o4SEpxUE002645 Authentication-Results: mail.leadmon.net; sender-id=pass header.from=howard@leadmon.net; auth=pass (LOGIN); spf=pass smtp.mfrom=howard@leadmon.net From: "Howard Leadmon" To: "'Jeremy Chadwick'" , "'Andriy Gapon'" References: <060401cafe37$a411b240$ec3516c0$@net><4BFF894F.4010008@icyb.net. ua> <20100528134549.GA75411@icarus.home.lan> In-Reply-To: <20100528134549.GA75411@icarus.home.lan> Date: Fri, 28 May 2010 10:51:59 -0400 Message-ID: <064701cafe75$537c1530$fa743f90$@net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Acr+bBxoGdlG04UPQlmLFByMWNN2YQABhuKA Content-Language: en-us X-Virus-Scanned: clamav-milter 0.96.1 at vorlon.leadmon.net X-Virus-Status: Clean Cc: amd64@freebsd.org, freebsd-fs@freebsd.org Subject: RE: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 14:52:01 -0000 > -----Original Message----- > From: Jeremy Chadwick [mailto:freebsd@jdc.parodius.com] > Sent: Friday, May 28, 2010 9:46 AM > To: Andriy Gapon > Cc: Howard Leadmon; amd64@freebsd.org; freebsd-fs@freebsd.org > Subject: Re: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. > > On Fri, May 28, 2010 at 12:13:51PM +0300, Andriy Gapon wrote: > > on 28/05/2010 10:30 Howard Leadmon said the following: > > > I know there used to be some issues with this a while back, but > thought > > > with the 8.x FBSD servers most of this tuned itself, or then again > maybe > > > this is something different. > > > > > > > > > > > > For the first time I ever recall, I found my FreeBSD 8 server was > crashed > > > this past morning, with the following error: > > > > > > > > > > > > panic:kmem_malloc(131072):kmem_map to small: 1296826368 total > allocated > > > > > > cupid=4 > > > > O wow, this is an amd64 system (with 4G RAM) and you've got "kmem_map > too > > small". That's very very strange. > > It is? On amd64, vm.kmem_size (not vm.kmem_size_max) is what has to be > increased. I can point folks to the "official" statement from pjd@ and > some others if need be. For a very long time I questioned this because > for an even longer amount of time we were being told to increase > vm.kmem_size_max. > > vm.kmem_size_max, by default, is already huge on amd64 (~320GB or > something like that). Proof: > > vm.kmem_size_max: 329853485875 > > To the OP: you will need to increase vm.kmem_size in /boot/loader.conf > and reboot the system. "What value do I pick?" With 4GB, I would > recommend you use these two variables: > > vm.kmem_size="2048M" > vfs.zfs.arc_max="1536M" > > This will increase the available kmem, and also limit the ARC size > explicitly to nothing larger than 1.5GB. This should stabilise your > system. > > -- > | Jeremy Chadwick jdc@parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP: 4BD6C0CB | Thanks Jeremy, I will try your recommended settings provided above. To the other poster, as to the settings of kmem, I had nothing specific set, just whatever FBSD was using by default. In loader.conf all I had was: zfs_load="YES" vfs.root.mountfrom="zfs:tank/root" As to the setting of kmem and arc, I had the following which I will assume were defaults or auto-tunes: vfs.zfs.arc_max : 862653440 vm.kmem_size : 1380245504 vm.kmem_size_max: 329853485875 Like Jeremy said, it sure looks like vm.kmem_size_max is quite large by default. After applying Jeremy's recommendations, I now see: vfs.zfs.arc_max : 1610612736 vm.kmem_size : 2147483648 vm.kmem_size_max: 329853485875 I guess while we are all on the subject, I notice in the dmesg log the message: ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present; to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. Is this anything I want to enable, that's like a big performance win, or do I just not have enough RAM to support it? Always been kinda curious about it, but so far I am liking ZFS, well outside of the machine panic.. LOL --- Howard Leadmon From owner-freebsd-fs@FreeBSD.ORG Fri May 28 15:51:51 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67544106566C for ; Fri, 28 May 2010 15:51:51 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.westchester.pa.mail.comcast.net (qmta09.westchester.pa.mail.comcast.net [76.96.62.96]) by mx1.freebsd.org (Postfix) with ESMTP id 131A98FC15 for ; Fri, 28 May 2010 15:51:50 +0000 (UTC) Received: from omta02.westchester.pa.mail.comcast.net ([76.96.62.19]) by qmta09.westchester.pa.mail.comcast.net with comcast id NyZX1e0060QuhwU593rrp2; Fri, 28 May 2010 15:51:51 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta02.westchester.pa.mail.comcast.net with comcast id P3ro1e00D3S48mS3N3rpNa; Fri, 28 May 2010 15:51:50 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 7C6E09B418; Fri, 28 May 2010 08:51:47 -0700 (PDT) Date: Fri, 28 May 2010 08:51:47 -0700 From: 'Jeremy Chadwick' To: Howard Leadmon Message-ID: <20100528155147.GA77427@icarus.home.lan> References: <060401cafe37$a411b240$ec3516c0$@net> <4BFF894F.4010008@icyb.net.ua> <20100528134549.GA75411@icarus.home.lan> <064701cafe75$537c1530$fa743f90$@net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <064701cafe75$537c1530$fa743f90$@net> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: amd64@freebsd.org, freebsd-fs@freebsd.org, 'Andriy Gapon' Subject: Re: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 15:51:51 -0000 On Fri, May 28, 2010 at 10:51:59AM -0400, Howard Leadmon wrote: > Thanks Jeremy, I will try your recommended settings provided above. > > To the other poster, as to the settings of kmem, I had nothing specific > set, just whatever FBSD was using by default. vfs.zfs.arc_max is calculated on-the-fly during ZFS module/init time, I believe, unless you explicitly set a value in loader.conf. vm.kmem_size is similar in that regard. I do not know the calculation formulas. vm.kmem_size_max is more or less static on amd64, because it represents the maximum amount of kmem usable/addressable. You can ignore the next few paragraphs if you don't care about the history of this tunable, but it will probably help folks reading the list. This is what I've figured out mostly on my own. Prior to February 2009 this value was significantly smaller due to VM design/implementation issues. Alan Cox (not the Linux guy) did the necessary work to fix this problem in RELENG_7 and committed things then. Fast forward to October 2009, by which time there were hundreds of posts from users/SAs talking about ZFS, stability problems, and the dreaded "kmem map is too small" error. I sent the following to -stable: http://lists.freebsd.org/pipermail/freebsd-stable/2009-October/052256.html The first thing you're going to notice is that I'm talking about RELENG_7, and specifically amd64. However, the exact same code/efforts (see above) was committed to RELENG_8 simultaneously (or within a very short period of time). So any RELENG_[78] amd64 system with sources from 2009/02 or later should have a very large vm.kmem_size_max. I can confirm this on the couple RELENG_7 systems we have in production. The second thing that you'll notice is one of the links in my mail: it points to a post from pjd@ stating that on amd64 you need to adjust vm.kmem_size, not vm.kmem_size_max. Take note of when this was said: September 2009. This was *after* Alan Cox's work, and I'm certain Pawel had that in mind. Fast forward to... I'm not sure what date; sometime in mid or late 2009. The behaviour of vfs.zfs.arc_max is changed so that it becomes a *hard limit* rather than a "high-water mark" like it was previously. I'm also not sure if this behaviour changed in just RELENG_8 or RELENG_7. My brain is full for a lot of different reasons; I try hard to remember as much as I can but it's too much for one person. Starting to see where all the confusion comes from? :-) Fast forward to today. People are still complaining about the problem, but when they do they usually don't provide enough details. Why? Because they don't know what details to provide. And why is that? Because people expect ZFS on FreeBSD to mimic Solaris 10 or OpenSolaris, where it "just works" (I know because we use it at my workplace on thousands of boxes). You tell users "well, you have to tune loader.conf" and they say "WHY?". You tell them what to tune and they ask "What values do I pick?", which vary from system to system and its workload. There's really no "magic number". Getting FreeBSD to that stage is difficult from what I understand (I believe John Baldwin and a few others have covered this topic). There are efforts underway to eventually solve this problem down the road. Anyway, until then -- I've offered this in the past and I'll offer it again: I'm 100% willing to sit down and write a document that could go into the Handbook that covers ZFS tuning on FreeBSD, why it's necessary (at this point in time), what values are needed, yadda yadda. But I can't write this for the same reason the ZFS section on the FreeBSD Wiki is outdated -- because to get answers to some of the questions, one needs the kernel folks working on this code to help provide answers. Most of us (myself included) are not familiar with the inner-workings of the ZFS port, nor are we fully familiar with that of the VM. The documentation dudes need the kernel dudes. :-) Back to the rest of your mail: > In loader.conf all I had was: > > zfs_load="YES" > vfs.root.mountfrom="zfs:tank/root" > > As to the setting of kmem and arc, I had the following which I will assume > were defaults or auto-tunes: > > vfs.zfs.arc_max : 862653440 > vm.kmem_size : 1380245504 > vm.kmem_size_max: 329853485875 > > {...below taken from your earlier mails...} > > panic:kmem_malloc(131072):kmem_map to small: 1296826368 total allocated I'm not entirely sure, but I think vfs.zfs.arc_max, if not explicitly set in loader.conf, might still act as a "high-water mark". Meaning, it's possible for the ZFS ARC to still exceed vm.kmem_size and cause a panic. Setting the arc_max value explicitly in loader.conf probably forces a hard limit, but I'm not sure. Can someone validate this? I'm basing it on the fact that 1,296,826,368 exceeds 862,653,440, and *probably* was attempting to exceed 1,380,245,504. What I do know is that by setting the two parameters I provided, I can bang on a RELENG_8 box and watch kstat.zfs.misc.arcstats.size never exceed vfs.zfs.arc_max, and the box never panics. All our systems in production, and my two at home, are tuned this way. > I guess while we are all on the subject, I notice in the dmesg log the > message: > > ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is > present; > to enable, add "vfs.zfs.prefetch_disable=0" to > /boot/loader.conf. > > Is this anything I want to enable, that's like a big performance win, or do > I just not have enough RAM to support it? Always been kinda curious about > it, but so far I am liking ZFS, well outside of the machine panic.. LOL And now you've touched on the *other* thing I've ranted about: how that message isn't accurate (nor have previous incarnations). Rather than explain it here, you can just read my blog entry about this message and hopefully what I've written will suffice for an explanation (see bottom half of the post). http://koitsu.wordpress.com/2009/10/12/testing-out-freebsd-8-0-rc1/ As for "should I actually enable this?" -- I've been in a private conversation with another FreeBSD user about this, and like me, he isn't sure either. Where did this arbitrary limit come from, and why are we being warned about it? Where can we read about the decision? This circles back to what I said earlier -- if documentation can't be provided, at bare minimum some explanations given in src/UPDATING would be sufficient. That's about all I can say on the matter. I do what I can, but the ability to accomplish what's needed is mostly out of my control. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Fri May 28 16:34:27 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B4A4106567D for ; Fri, 28 May 2010 16:34:27 +0000 (UTC) (envelope-from james-freebsd-fs2@jrv.org) Received: from mail.jrv.org (rrcs-24-73-246-106.sw.biz.rr.com [24.73.246.106]) by mx1.freebsd.org (Postfix) with ESMTP id BB97C8FC1B for ; Fri, 28 May 2010 16:34:26 +0000 (UTC) Received: from kremvax.housenet.jrv (kremvax.housenet.jrv [192.168.3.124]) by mail.jrv.org (8.14.3/8.14.3) with ESMTP id o4SGYNCn035792; Fri, 28 May 2010 11:34:23 -0500 (CDT) (envelope-from james-freebsd-fs2@jrv.org) Authentication-Results: mail.jrv.org; domainkeys=pass (testing) header.from=james-freebsd-fs2@jrv.org DomainKey-Signature: a=rsa-sha1; s=enigma; d=jrv.org; c=nofws; q=dns; h=message-id:date:from:user-agent:mime-version:cc:subject: references:in-reply-to:content-type:content-transfer-encoding; b=CUUXquYN2+MZySwbRxMdFKvdC9nhPNAsbEqWBZwHaEeHGZ1fEsmWGMOW1mi/MS8p5 ldFF+zkU7zbFTgqywHTCGAwzujKH68C9IaZ6woCatgh2WFpbP1kG1y2C0YkBoS9C/bf 5G7CaObQXVWpUxpdiMh+VWjeHQXX61E2cIWq3e4= Message-ID: <4BFFF08F.3060103@jrv.org> Date: Fri, 28 May 2010 11:34:23 -0500 From: "James R. Van Artsdalen" User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 References: <4BFFC6E6.2070505@jrv.org> <20100528134926.GB75411@icarus.home.lan> In-Reply-To: <20100528134926.GB75411@icarus.home.lan> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs Subject: Re: ZFS: how to replace a dead disk? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 16:34:27 -0000 Jeremy Chadwick wrote: > On Fri, May 28, 2010 at 08:36:38AM -0500, James R. Van Artsdalen wrote: >> What's the right way to replace a dead disk under ZFS? >> >> replacing DEGRADED 0 0 0 >> ada1/old UNAVAIL 0 256K 0 cannot open >> ada1 ONLINE 0 0 0 1.47T resilvered >> --- >> >> It says "replacing" and that the device, vdev and pool are degraded, yet >> the "resilver" finished hours ago. I cannot detach the ada1/old entry. >> >> Is there some other command I should have used to remove the dead ada1 >> device? > > What version of FreeBSD? Please provide uname -a output and not "8.0" > or something equally as terse. > > Some clarification: you didn't remove the device, you simply told ZFS to > assuming that the device had been replaced. > > What did you do (both physically and software/command-line-wise) *prior* > to issuing "zpool replace jwrc ada1"? > Sorry: my original note contained version information but that isn't in your reply? FreeBSD cyclone 9.0-CURRENT FreeBSD 9.0-CURRENT #2 r206111: Fri Apr 2 13:47:20 CDT 2010 root@cyclone.housenet.jrv:/usr/obj/usr/src/sys/GENERIC amd64 The original disk is no longer usable by FreeBSD in any way: it returned a stream of errors & noise on its port in a way that left the system unable to boot. I physically replaced that disk with a new disk before attempting the "zpool replace" No actions were taken prior to replacing the disk. I went to the site to see why the server was unresponsive, saw that one drive was problematic by watching the activity LEDs, physically replaced that disk, booted, and logically replaced that disk with "zpool replace jwrc ada1" From owner-freebsd-fs@FreeBSD.ORG Fri May 28 16:49:44 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E040106564A; Fri, 28 May 2010 16:49:44 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 2D4468FC1C; Fri, 28 May 2010 16:49:42 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA15166; Fri, 28 May 2010 19:49:27 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4BFFF416.1080901@icyb.net.ua> Date: Fri, 28 May 2010 19:49:26 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100517) MIME-Version: 1.0 To: Jeremy Chadwick References: <060401cafe37$a411b240$ec3516c0$@net> <4BFF894F.4010008@icyb.net.ua> <20100528134549.GA75411@icarus.home.lan> In-Reply-To: <20100528134549.GA75411@icarus.home.lan> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: amd64@freebsd.org, freebsd-fs@freebsd.org Subject: Re: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 16:49:44 -0000 on 28/05/2010 16:45 Jeremy Chadwick said the following: > It is? On amd64, vm.kmem_size (not vm.kmem_size_max) is what has to be > increased. I can point folks to the "official" statement from pjd@ and > some others if need be. For a very long time I questioned this because > for an even longer amount of time we were being told to increase > vm.kmem_size_max. > > vm.kmem_size_max, by default, is already huge on amd64 (~320GB or > something like that). Proof: > > vm.kmem_size_max: 329853485875 > > To the OP: you will need to increase vm.kmem_size in /boot/loader.conf > and reboot the system. "What value do I pick?" With 4GB, I would > recommend you use these two variables: > > vm.kmem_size="2048M" > vfs.zfs.arc_max="1536M" > > This will increase the available kmem, and also limit the ARC size > explicitly to nothing larger than 1.5GB. This should stabilise your > system. > Jeremy, you are correct, I was confused about what kmem_size_max and kmem_size meant. kmem_size_max is what could be available, in principle, for kmem_map. But in practice kmem_map size is auto-set to about 1/3 of physical memory available, this is on amd64. BTW, the "6GB" change you mentioned in your other email was more relevant to kmem_size_max, it was overridden since then with 512GB change. kmem size calculations are very well explained here: http://www.mailinglistarchive.com/freebsd-current@freebsd.org/msg10986.html BTW, I don't see why we set kmem size to a fraction of physical memory on amd64 where we have plenty of KVM available. I don't buy the argument that larger kmem size would allow kernel to exhaust all physical memory which would lead to a "bad thing", e.g. a panic - right now kernel can exhaust all kmem which does lead to a panic. Perhaps, default value of VM_KMEM_SIZE_SCALE should be change to one on amd64? -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Fri May 28 17:11:30 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09F191065673 for ; Fri, 28 May 2010 17:11:30 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta14.emeryville.ca.mail.comcast.net (qmta14.emeryville.ca.mail.comcast.net [76.96.27.212]) by mx1.freebsd.org (Postfix) with ESMTP id E4D6C8FC1D for ; Fri, 28 May 2010 17:11:29 +0000 (UTC) Received: from omta22.emeryville.ca.mail.comcast.net ([76.96.30.89]) by qmta14.emeryville.ca.mail.comcast.net with comcast id P0J41e0041vN32cAE5BVFH; Fri, 28 May 2010 17:11:29 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta22.emeryville.ca.mail.comcast.net with comcast id P5BV1e0013S48mS8i5BVd1; Fri, 28 May 2010 17:11:29 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 019109B419; Fri, 28 May 2010 10:11:29 -0700 (PDT) Date: Fri, 28 May 2010 10:11:28 -0700 From: Jeremy Chadwick To: "James R. Van Artsdalen" Message-ID: <20100528171128.GA79702@icarus.home.lan> References: <4BFFC6E6.2070505@jrv.org> <20100528134926.GB75411@icarus.home.lan> <4BFFF08F.3060103@jrv.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BFFF08F.3060103@jrv.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs Subject: Re: ZFS: how to replace a dead disk? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 17:11:30 -0000 On Fri, May 28, 2010 at 11:34:23AM -0500, James R. Van Artsdalen wrote: > Jeremy Chadwick wrote: > > On Fri, May 28, 2010 at 08:36:38AM -0500, James R. Van Artsdalen wrote: > >> What's the right way to replace a dead disk under ZFS? > >> > >> replacing DEGRADED 0 0 0 > >> ada1/old UNAVAIL 0 256K 0 cannot open > >> ada1 ONLINE 0 0 0 1.47T resilvered > >> --- > >> > >> It says "replacing" and that the device, vdev and pool are degraded, yet > >> the "resilver" finished hours ago. I cannot detach the ada1/old entry. > >> > >> Is there some other command I should have used to remove the dead ada1 > >> device? > > > > What version of FreeBSD? Please provide uname -a output and not "8.0" > > or something equally as terse. > > > > Some clarification: you didn't remove the device, you simply told ZFS to > > assuming that the device had been replaced. > > > > What did you do (both physically and software/command-line-wise) *prior* > > to issuing "zpool replace jwrc ada1"? > > > Sorry: my original note contained version information but that isn't in > your reply? > > FreeBSD cyclone 9.0-CURRENT FreeBSD 9.0-CURRENT #2 r206111: Fri Apr 2 > 13:47:20 CDT 2010 > root@cyclone.housenet.jrv:/usr/obj/usr/src/sys/GENERIC amd64 The only line sent to the list was: "This kernel 206111, roughly April 1, on amd64". Here's verification: http://lists.freebsd.org/pipermail/freebsd-fs/2010-May/008592.html So now we know you're running HEAD. > The original disk is no longer usable by FreeBSD in any way: it returned > a stream of errors & noise on its port in a way that left the system > unable to boot. I physically replaced that disk with a new disk before > attempting the "zpool replace" > > No actions were taken prior to replacing the disk. I went to the site > to see why the server was unresponsive, saw that one drive was > problematic by watching the activity LEDs, physically replaced that > disk, booted, and logically replaced that disk with "zpool replace jwrc > ada1" I think the procedure you executed might be the problem. The steps I've used in the past, 100% reliably, with ata(4) and AHCI on an ICH7 and ICH9 are: 1. zpool offline adX 2. atacontrol list (to find the ataX device number) 3. atacontrol detach ataX 4. dmesg (verify the detach worked) 5. Physically remove the disk (must be in a hot-swap enclosure) 6. Physically insert the new disk 7. atacontrol attach ataX 8. dmesg (to determine what the adX drive number is; on my systems the adX drive number remains static/does not change) 9. zpool online pool adX 10. zpool replace pool adX 11. zpool status (watch until finished) This is adherent to the Solaris ZFS Administrator's guide, except that atacontrol(8) is being used instead of cfgadm(1M). See Example 11-1: http://docs.sun.com/app/docs/doc/819-5461/gbbzy?l=en&a=view The same procedure should ideally be followed using ahci.ko + CAM, using camcontrol devlist/eject, camcontrol rescan (may not be needed but use devlist to verify the kernel noticing removal/additions), and camcontrol load. If you'd like me to verify and demonstrate this on FreeBSD (RELENG_8 only, however -- I don't run CURRENT) I can do so. I can also do the same thing with ahci.ko + CAM. Just let me know. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Fri May 28 18:51:23 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C68C106574C for ; Fri, 28 May 2010 18:51:23 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 33D348FC08 for ; Fri, 28 May 2010 18:51:23 +0000 (UTC) Received: from volatile.chemikals.org (adsl-67-123-29.shv.bellsouth.net [98.67.123.29]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by warped.bluecherry.net (Postfix) with ESMTPSA id 15F89879023E; Fri, 28 May 2010 13:51:21 -0500 (CDT) Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.14.4/8.14.4) with ESMTP id o4SIpITu029351; Fri, 28 May 2010 13:51:18 -0500 (CDT) (envelope-from morganw@chemikals.org) Date: Fri, 28 May 2010 13:51:17 -0500 (CDT) From: Wes Morgan X-X-Sender: morganw@volatile To: "James R. Van Artsdalen" In-Reply-To: <4BFFF08F.3060103@jrv.org> Message-ID: References: <4BFFC6E6.2070505@jrv.org> <20100528134926.GB75411@icarus.home.lan> <4BFFF08F.3060103@jrv.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: clamav-milter 0.95.3 at warped X-Virus-Status: Clean Cc: freebsd-fs Subject: Re: ZFS: how to replace a dead disk? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2010 18:51:23 -0000 On Fri, 28 May 2010, James R. Van Artsdalen wrote: > Jeremy Chadwick wrote: > > On Fri, May 28, 2010 at 08:36:38AM -0500, James R. Van Artsdalen wrote: > >> What's the right way to replace a dead disk under ZFS? > >> > >> replacing DEGRADED 0 0 0 > >> ada1/old UNAVAIL 0 256K 0 cannot open > >> ada1 ONLINE 0 0 0 1.47T resilvered > >> --- > >> > >> It says "replacing" and that the device, vdev and pool are degraded, yet > >> the "resilver" finished hours ago. I cannot detach the ada1/old entry. > >> > >> Is there some other command I should have used to remove the dead ada1 > >> device? > > > > What version of FreeBSD? Please provide uname -a output and not "8.0" > > or something equally as terse. > > > > Some clarification: you didn't remove the device, you simply told ZFS to > > assuming that the device had been replaced. > > > > What did you do (both physically and software/command-line-wise) *prior* > > to issuing "zpool replace jwrc ada1"? > > > Sorry: my original note contained version information but that isn't in > your reply? > > FreeBSD cyclone 9.0-CURRENT FreeBSD 9.0-CURRENT #2 r206111: Fri Apr 2 > 13:47:20 CDT 2010 > root@cyclone.housenet.jrv:/usr/obj/usr/src/sys/GENERIC amd64 > > The original disk is no longer usable by FreeBSD in any way: it returned > a stream of errors & noise on its port in a way that left the system > unable to boot. I physically replaced that disk with a new disk before > attempting the "zpool replace" > > No actions were taken prior to replacing the disk. I went to the site > to see why the server was unresponsive, saw that one drive was > problematic by watching the activity LEDs, physically replaced that > disk, booted, and logically replaced that disk with "zpool replace jwrc > ada1" Is it possible your array had some errors on other devices prior to the ada1 disk failing? When was the last scrub?