From owner-freebsd-fs@FreeBSD.ORG Mon Oct 22 11:07:05 2007 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 19A9F16A41A for ; Mon, 22 Oct 2007 11:07:05 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 08A4413C48D for ; Mon, 22 Oct 2007 11:07:05 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.1/8.14.1) with ESMTP id l9MB74f1079931 for ; Mon, 22 Oct 2007 11:07:04 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.1/8.14.1/Submit) id l9MB745t079927 for freebsd-fs@FreeBSD.org; Mon, 22 Oct 2007 11:07:04 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 22 Oct 2007 11:07:04 GMT Message-Id: <200710221107.l9MB745t079927@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2007 11:07:05 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o kern/114856 fs [ntfs] [patch] Bug in NTFS allows bogus file modes. o kern/116170 fs Kernel panic when mounting /tmp 4 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/114847 fs [ntfs] [patch] dirmask support for NTFS ala MSDOSFS 1 problem total. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 22 15:35:22 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD6D116A420 for ; Mon, 22 Oct 2007 15:35:22 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 6C44D13C48A for ; Mon, 22 Oct 2007 15:35:22 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id C40F423C466; Mon, 22 Oct 2007 17:35:21 +0200 (CEST) Date: Mon, 22 Oct 2007 17:35:21 +0200 From: Peter Schuller To: freebsd-fs@freebsd.org Message-ID: <20071022153521.GB27594@hyperion.scode.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WhfpMioaduB5tiZL" Content-Disposition: inline User-Agent: Mutt/1.5.16 (2007-06-09) Subject: ZFS: reproducable inability to accesss a pool (process hangs; other pools fine) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2007 15:35:22 -0000 --WhfpMioaduB5tiZL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, On the same system I recently posted about on -stable, with RELENG_7 =66rom a few days ago, I am now running a SiL 3114 on a raidz2 in degraded mode with one disk missing (it is degraded by design because I wanted to create a 5 disk array but only had 4). For the purpose of discovering any stability issues with the 3114 controller I did some stress tests that have yet to reveil controller problems, but has triggered what appears to be a ZFS problem. Test case: /promraid - root of the pool in question /promraid/ports - copy of /usr/ports tree from my machine /promraid/1 - empty directory /promraid/2 - empty directory I now run concurrently in two shells: while [ 1 ] ; do rsync -a /promraid/ports /promraid/1/pp ; rm -rf /promraid= /1/pp ; done and: while [ 1 ] ; do rsync -a /promraid/ports /promraid/2/pp ; rm -rf /promraid= /2/pp ; done This runs fine for some hours, but eventually I end up with hung rsyncs in "zfs" state according to op. Attempting to e.g. ls /promraid hangs as well. Yet ZFS continues working (another pool is entirely fine), and there are no errors in dmesg. iostat -x does NOT indicate that it is perpetually waiting on I/O from a disk or something likethat (0% utilization). The processes are unkillable, even by SIGKILL. I should have this environment for a few more days, so can hopefully reproduce this again. It has happened at least twice already (the first time I was in X and X hung; I thought I had a panic so re-ran the tests in the console; these two times I didn't get a panic but I am unsure whether the failure case is different). Does anyone have suggestions for what to do to produce the best information possible? Given that there are no errors, no panic, etc. One obvious bit is to ktrace them I realize, if that gives me anything (the size of the trace if I were to trace it from the beginning would, I suspect, be prohibitive). Will do that next time. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --WhfpMioaduB5tiZL Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHHMM4DNor2+l1i30RApmKAKCjtvR5O6TIh7RBFderKc1cZElg3gCdFIMm bFT0M9YWhc5avTYUxnhI3uw= =qSJW -----END PGP SIGNATURE----- --WhfpMioaduB5tiZL-- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 22 16:00:41 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAC1A16A420; Mon, 22 Oct 2007 16:00:41 +0000 (UTC) (envelope-from lulf@stud.ntnu.no) Received: from fri.itea.ntnu.no (fri.itea.ntnu.no [129.241.7.60]) by mx1.freebsd.org (Postfix) with ESMTP id 7E5C813C48E; Mon, 22 Oct 2007 16:00:41 +0000 (UTC) (envelope-from lulf@stud.ntnu.no) Received: from localhost (localhost [127.0.0.1]) by fri.itea.ntnu.no (Postfix) with ESMTP id 241468E09; Mon, 22 Oct 2007 18:00:31 +0200 (CEST) Received: from caracal.stud.ntnu.no (caracal.stud.ntnu.no [129.241.56.185]) by fri.itea.ntnu.no (Postfix) with ESMTP; Mon, 22 Oct 2007 18:00:30 +0200 (CEST) Received: by caracal.stud.ntnu.no (Postfix, from userid 2312) id 50A4C624102; Mon, 22 Oct 2007 18:00:41 +0200 (CEST) Date: Mon, 22 Oct 2007 18:00:41 +0200 From: Ulf Lilleengen To: Kris Kennaway Message-ID: <20071022160041.GA12710@stud.ntnu.no> References: <20070816100526.GA31897@stud.ntnu.no> <20071017220948.GA4279@stud.ntnu.no> <47170C54.6050502@FreeBSD.org> <20071018083448.GA1079@stud.ntnu.no> <47171D89.7030906@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <47171D89.7030906@FreeBSD.org> User-Agent: Mutt/1.5.9i X-Content-Scanned: with sophos and spamassassin at mailgw.ntnu.no. Cc: freebsd-fs@freebsd.org Subject: Re: [PATCH] Make fdescfs MPSAFE X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2007 16:00:41 -0000 On tor, okt 18, 2007 at 10:47:05 +0200, Kris Kennaway wrote: > Ulf Lilleengen wrote: > >On tor, okt 18, 2007 at 09:33:40 +0200, Kris Kennaway wrote: > >>Ulf Lilleengen wrote: > >>>On tor, aug 16, 2007 at 12:05:26 +0200, Ulf Lilleengen wrote: > >>>>Hi, > >>>> [...] > >>>http://folk.ntnu.no/lulf/patches/freebsd/fdescfs/fdescfs_lock.diff > >>This might be OK but you should be aware that rwlocks can be slower than > >>mutexes when there is a suitably mixed read/write workload. We don't do > >>the same adaptive spinning for wlocks as for mutexes when they are held > >>by shared holders (since we don't track who they are so can't track > >>whether they're running), and it is possible for readers to starve > >>writers. > >> > >>If possible some benchmarks trying to find the worst case behaviour > >>would be useful. > >Good point. I guess having a benchmark where one would open #hashentries > >files, and then have #hashentries threads reading (accessing the files) and > >one thread writing (closing perhaps) could produce the starvation? > > Perhaps, give it a try :) > I created the program found here: http://folk.ntnu.no/lulf/dev/freebsd/fdescfs/benchmark.c I compiled it with -lthr, and the hardware I've tested it on is a core duo, as well as a dual-cpu p3 machine. I tried opening X files, then having X "readers" doing a lookup on the filedescriptors for those files in /dev/fd (Doing a lookup for a first time puts it into the hash table (using wlock). Looking it up the second time, it will just use rlock). While these X readers was busy reading, tried to open a new file and doing lookup on it (will try to insert and acquire wlock). I tried with X = 1, 2, 4 and 16. I was unable to produce the starvation with this. I'll try see if I can come up with some other benchmark. I assume it would be easier to reproduce it if I have CPUs I can run threads on to assure I always hold it, but I don't. Also, as a general way to observe behaviour in benchmarks, is there a way I could force/check what CPU a thread is running on from userland? -- Ulf Lilleengen From owner-freebsd-fs@FreeBSD.ORG Tue Oct 23 07:21:21 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD88916A417; Tue, 23 Oct 2007 07:21:21 +0000 (UTC) (envelope-from daichi@freebsd.org) Received: from natial.ongs.co.jp (natial.ongs.co.jp [202.216.232.58]) by mx1.freebsd.org (Postfix) with ESMTP id E0B4C13C48D; Tue, 23 Oct 2007 07:21:20 +0000 (UTC) (envelope-from daichi@freebsd.org) Received: from parancell.ongs.co.jp (dullmdaler.ongs.co.jp [202.216.232.62]) by natial.ongs.co.jp (Postfix) with ESMTP id 62318244C48; Tue, 23 Oct 2007 13:51:44 +0900 (JST) Message-ID: <471D7DE0.2070103@freebsd.org> Date: Tue, 23 Oct 2007 13:51:44 +0900 From: Daichi GOTO User-Agent: Thunderbird 2.0.0.6 (X11/20070803) MIME-Version: 1.0 To: FreeBSD Current , freebsd-stable@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Daichi GOTO , Ed Schouten , Stanislav Sedov , Jeff Roberson , Ken Smith Subject: [ANN] 8-CURRENT, RELENG_7 and RELENG_6 have gotten latest unionfs improvements X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2007 07:21:21 -0000 Hi unionfs folks It is my pleasure and honor to announce the commitment of latest unionfs improvements for 8-current, RELENG_7 and RELENG_6. Now you can get more stable operation using unionfs on latest 8/7/6. This latest improvements give finstall and FreeSBIE works more well as well as other unionfs works good. If you have interesting in it, would you try it please. However, follow two issues we still have: - It gets a hang-up in a certain case using with NFS. - We cannot see devfs on unionfs. I am going to try those issue fix in near future. Thanks all folks who have helped us! The documents of those unionfs: http://people.freebsd.org/~daichi/unionfs/ (English) http://people.freebsd.org/~daichi/unionfs/index-ja.html (Japanese) -- Daichi GOTO, http://people.freebsd.org/~daichi From owner-freebsd-fs@FreeBSD.ORG Tue Oct 23 23:55:33 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7995316A418; Tue, 23 Oct 2007 23:55:33 +0000 (UTC) (envelope-from amdmi3@amdmi3.ru) Received: from cp65.agava.net (cp65.agava.net [89.108.66.215]) by mx1.freebsd.org (Postfix) with ESMTP id F33CA13C4A5; Tue, 23 Oct 2007 23:55:32 +0000 (UTC) (envelope-from amdmi3@amdmi3.ru) Received: from [213.148.20.85] (helo=nexii.panopticon) by cp65.agava.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.44 (FreeBSD)) id 1IkTaL-000EkV-DP; Wed, 24 Oct 2007 03:54:57 +0400 Received: from hades.panopticon (hades.panopticon [192.168.0.2]) by nexii.panopticon (Postfix) with ESMTP id 602A717049; Wed, 24 Oct 2007 03:55:51 +0400 (MSD) Received: by hades.panopticon (Postfix, from userid 1000) id 170E7415B; Wed, 24 Oct 2007 03:56:32 +0400 (MSD) Date: Wed, 24 Oct 2007 03:56:32 +0400 From: Dmitry Marakasov To: Daichi GOTO Message-ID: <20071023235632.GB78191@hades.panopticon> Mail-Followup-To: Daichi GOTO , FreeBSD Current , freebsd-stable@freebsd.org, freebsd-fs@freebsd.org, Masanori OZAWA , Ed Schouten , Stanislav Sedov , Jeff Roberson , Ken Smith References: <471D7DE0.2070103@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <471D7DE0.2070103@freebsd.org> User-Agent: Mutt/1.5.16 (2007-06-09) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - cp65.agava.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [26 6] X-AntiAbuse: Sender Address Domain - amdmi3.ru X-Source: X-Source-Args: X-Source-Dir: Cc: Stanislav Sedov , freebsd-stable@freebsd.org, Ed Schouten , freebsd-fs@freebsd.org, FreeBSD Current , Jeff Roberson , Ken Smith Subject: Re: [ANN] 8-CURRENT, RELENG_7 and RELENG_6 have gotten latest unionfs improvements X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2007 23:55:33 -0000 * Daichi GOTO (daichi@freebsd.org) wrote: > It is my pleasure and honor to announce the commitment of > latest unionfs improvements for 8-current, RELENG_7 and > RELENG_6. Now you can get more stable operation using > unionfs on latest 8/7/6. > > This latest improvements give finstall and FreeSBIE works > more well as well as other unionfs works good. If you have > interesting in it, would you try it please. Thanks a lot all your guys for your work! Btw, I was wondering: is unionfs related in any way with -ounion option of mount_*? I was told long time ago that -ounion is even more broken than unionfs. Though, those two features seem to do very similar thing and I think that -ounion option is pretty useful. -- Dmitry A. Marakasov | jabber: amdmi3@jabber.ru amdmi3@amdmi3.ru | http://www.amdmi3.ru From owner-freebsd-fs@FreeBSD.ORG Wed Oct 24 13:16:54 2007 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3AEA116A421; Wed, 24 Oct 2007 13:16:54 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [IPv6:2001:1b20:1:3::1]) by mx1.freebsd.org (Postfix) with ESMTP id A5CD613C4A5; Wed, 24 Oct 2007 13:16:53 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.1/8.14.1) with ESMTP id l9ODGYXJ093702; Wed, 24 Oct 2007 15:16:41 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.1/8.14.1/Submit) id l9ODGYfZ093701; Wed, 24 Oct 2007 15:16:34 +0200 (CEST) (envelope-from olli) Date: Wed, 24 Oct 2007 15:16:34 +0200 (CEST) Message-Id: <200710241316.l9ODGYfZ093701@lurza.secnetix.de> From: Oliver Fromme To: freebsd-stable@FreeBSD.ORG, freebsd-fs@FreeBSD.ORG, freebsd-current@FreeBSD.ORG, amdmi3@amdmi3.ru In-Reply-To: X-Newsgroups: list.freebsd-stable User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (FreeBSD/6.2-STABLE-20070808 (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Wed, 24 Oct 2007 15:16:42 +0200 (CEST) Cc: Subject: Re: [ANN] 8-CURRENT, RELENG_7 and RELENG_6 have gotten latest ?unionfs improvements X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-stable@FreeBSD.ORG, freebsd-fs@FreeBSD.ORG, freebsd-current@FreeBSD.ORG, amdmi3@amdmi3.ru List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2007 13:16:54 -0000 [Note: stripped excessive Cc.] Dmitry Marakasov wrote: > Btw, I was wondering: is unionfs related in any way with -ounion > option of mount_*? No. The union mount option (-o union) is completely separate from UNIONFS, although it can be used to achieve a somewhat similar effect. It depends on your requirements whether it is sufficient or not. > I was told long time ago that -ounion is even > more broken than unionfs. That's wrong. The union mount option was _never_ really broken. I'm using it for almost as long as FreeBSD exists. However UNIONFS was broken for a long time (along with NULLFS and UMAPFS). NULLFS has been fixed some time ago, UMAPFS was abandoned apparently, i.e. nobody showed up to fix it, and UNIONFS was pretty much completely overhauled by Daichi GOTO and his team. I would now regard it as stable. > Though, those two features seem to do very > similar thing and I think that -ounion option is pretty useful. Yes, it is useful. The biggest differences are: - The union mount option newly mounts a filesystem on top of an arbitrary existing directory tree, while UNIONFS mounts another representation of one existing directory tree on top of another one. That means UNIONFS does the same as NULLFS, but unlike NULLFS it does not hide the underlying directory tree. - When using the union mount option, only the entries in the root directory show through from the "lower" file system. When using UIONFS, _all_ entries in _all_ directories are visible (unless masked by an identical entry in the upper file system, of course). - The implementation of the union mount option is rather simple has rather low overhead. UNIONFS is much more complex and has some overhead for certain operations, especially when files and directories have to be created automatically in the upper layer. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Python is an experiment in how much freedom programmers need. Too much freedom and nobody can read another's code; too little and expressiveness is endangered." -- Guido van Rossum From owner-freebsd-fs@FreeBSD.ORG Wed Oct 24 15:18:59 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C88B16A51E for ; Wed, 24 Oct 2007 15:18:59 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 0875513C4AA for ; Wed, 24 Oct 2007 15:18:58 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from localhost (localhost [127.0.0.1]) by people.fsn.hu (Postfix) with ESMTP id 8C46C84437 for ; Wed, 24 Oct 2007 16:48:43 +0200 (CEST) Received: from people.fsn.hu ([127.0.0.1]) by localhost (people.fsn.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 71641-03 for ; Wed, 24 Oct 2007 16:48:38 +0200 (CEST) Received: from japan.t-online.private (unknown [192.168.2.3]) by people.fsn.hu (Postfix) with ESMTP id C3CA484429 for ; Wed, 24 Oct 2007 16:48:38 +0200 (CEST) Message-ID: <471F5B46.9050106@fsn.hu> Date: Wed, 24 Oct 2007 16:48:38 +0200 From: Attila Nagy User-Agent: Thunderbird 2.0.0.0 (X11/20070421) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at fsn.hu Subject: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2007 15:18:59 -0000 Hello, I have an experimental (but that does not mean, I wouldn't like to get my data back :) zpool, which was created with something like this: zpool create people raidz2 /dev/da0 /dev/da3 /dev/da4, etc The problem is those device names have been changed during the next reboot (the cause of this is irrelevant, but mainly because some of them were not attached at the original boot, just later, so at the next reboot the disks came up in a different order), so now I have: zpool status pool: people state: UNAVAIL status: One or more devices could not be opened. There are insufficient replicas for the pool to continue functioning. action: Attach the missing device and online it using 'zpool online'. see: http://www.sun.com/msg/ZFS-8000-D3 scrub: none requested config: NAME STATE READ WRITE CKSUM people UNAVAIL 0 0 0 insufficient replicas raidz2 UNAVAIL 0 0 0 insufficient replicas da0 UNAVAIL 0 0 0 cannot open da3 ONLINE 0 0 0 da4 FAULTED 0 0 0 corrupted data da5 FAULTED 0 0 0 corrupted data da6 FAULTED 0 0 0 corrupted data da7 FAULTED 0 0 0 corrupted data da8 FAULTED 0 0 0 corrupted data da9 FAULTED 0 0 0 corrupted data (it seems da3 is still da3 :) My question is: what now? Is it possible to regain the pool, or is it totally busted now? I am not sure that I can figure out which device is which now... I've only played with ZFS on Solaris with FC targets, and there I've never faced this problem, because of the static naming. ps: I guess next time I will use glabel -I love that- to provide base devices... Thanks, -- Attila Nagy e-mail: Attila.Nagy@fsn.hu Free Software Network (FSN.HU) phone: +3630 306 6758 http://www.fsn.hu/ From owner-freebsd-fs@FreeBSD.ORG Wed Oct 24 15:26:34 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D7BF16A418 for ; Wed, 24 Oct 2007 15:26:34 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 702FE13C4A5 for ; Wed, 24 Oct 2007 15:26:34 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 24ED123C446; Wed, 24 Oct 2007 17:26:33 +0200 (CEST) Date: Wed, 24 Oct 2007 17:26:33 +0200 From: Peter Schuller To: Attila Nagy Message-ID: <20071024152632.GA85583@hyperion.scode.org> References: <471F5B46.9050106@fsn.hu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="T4sUOijqQbZv57TR" Content-Disposition: inline In-Reply-To: <471F5B46.9050106@fsn.hu> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2007 15:26:34 -0000 --T4sUOijqQbZv57TR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > I have an experimental (but that does not mean, I wouldn't like to get my= =20 > data back :) zpool, which was created with something like this: > zpool create people raidz2 /dev/da0 /dev/da3 /dev/da4, etc [snip] > My question is: what now? Is it possible to regain the pool, or is it=20 > totally busted now? I am not sure that I can figure out which device is= =20 > which now... I believe a re-import of the pool will solve this problem. > ps: I guess next time I will use glabel -I love that- to provide base=20 > devices... That's what I do. Even if I don't think it's needed I *always* glabel because I always end up in some wonky situation where I would have been less happy if I hadn't ;) --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --T4sUOijqQbZv57TR Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHH2QoDNor2+l1i30RAgccAKDcwjAnZH9Eezjqe9rmF5JlQk7ZdACgjuWI ac4zAavhE6zEW0UXqS4F0ys= =6jeC -----END PGP SIGNATURE----- --T4sUOijqQbZv57TR-- From owner-freebsd-fs@FreeBSD.ORG Wed Oct 24 15:35:36 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CEDE016A417 for ; Wed, 24 Oct 2007 15:35:36 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 98D5F13C494 for ; Wed, 24 Oct 2007 15:35:36 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from localhost (localhost [127.0.0.1]) by people.fsn.hu (Postfix) with ESMTP id 29F8884423 for ; Wed, 24 Oct 2007 17:35:23 +0200 (CEST) Received: from people.fsn.hu ([127.0.0.1]) by localhost (people.fsn.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 74543-09 for ; Wed, 24 Oct 2007 17:35:15 +0200 (CEST) Received: from japan.t-online.private (unknown [192.168.2.3]) by people.fsn.hu (Postfix) with ESMTP id 4CB5284420 for ; Wed, 24 Oct 2007 17:35:15 +0200 (CEST) Message-ID: <471F6632.5070506@fsn.hu> Date: Wed, 24 Oct 2007 17:35:14 +0200 From: Attila Nagy User-Agent: Thunderbird 2.0.0.0 (X11/20070421) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <471F5B46.9050106@fsn.hu> In-Reply-To: <471F5B46.9050106@fsn.hu> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at fsn.hu Subject: Re: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2007 15:35:36 -0000 On 10/24/07 16:48, Attila Nagy wrote: > Hello, > > I have an experimental (but that does not mean, I wouldn't like to get > my data back :) zpool, which was created with something like this: > zpool create people raidz2 /dev/da0 /dev/da3 /dev/da4, etc > > The problem is those device names have been changed during the next > reboot (the cause of this is irrelevant, but mainly because some of > them were not attached at the original boot, just later, so at the > next reboot the disks came up in a different order), so now I have: > zpool status > pool: people > state: UNAVAIL > status: One or more devices could not be opened. There are insufficient > replicas for the pool to continue functioning. > action: Attach the missing device and online it using 'zpool online'. > see: http://www.sun.com/msg/ZFS-8000-D3 > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > people UNAVAIL 0 0 0 insufficient replicas > raidz2 UNAVAIL 0 0 0 insufficient replicas > da0 UNAVAIL 0 0 0 cannot open > da3 ONLINE 0 0 0 > da4 FAULTED 0 0 0 corrupted data > da5 FAULTED 0 0 0 corrupted data > da6 FAULTED 0 0 0 corrupted data > da7 FAULTED 0 0 0 corrupted data > da8 FAULTED 0 0 0 corrupted data > da9 FAULTED 0 0 0 corrupted data > > (it seems da3 is still da3 :) > > My question is: what now? Is it possible to regain the pool, or is it > totally busted now? I am not sure that I can figure out which device > is which now... > > I've only played with ZFS on Solaris with FC targets, and there I've > never faced this problem, because of the static naming. > > ps: I guess next time I will use glabel -I love that- to provide base > devices... > > Thanks, > I reply to myself: rm /boot/zfs/zpool.cache reboot (I don't know the internals, maybe it can be solved without that) zfs import people -- Attila Nagy e-mail: Attila.Nagy@fsn.hu Free Software Network (FSN.HU) phone: +3630 306 6758 http://www.fsn.hu/ From owner-freebsd-fs@FreeBSD.ORG Wed Oct 24 15:58:18 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7918416A417 for ; Wed, 24 Oct 2007 15:58:18 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id DCD5213C491 for ; Wed, 24 Oct 2007 15:58:17 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4267C45F66; Wed, 24 Oct 2007 17:58:07 +0200 (CEST) Received: from localhost (public-gprs20687.centertel.pl [87.96.80.207]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 8507A456AB; Wed, 24 Oct 2007 17:57:52 +0200 (CEST) Date: Wed, 24 Oct 2007 17:57:23 +0200 From: Pawel Jakub Dawidek To: Attila Nagy Message-ID: <20071024155723.GA1431@garage.freebsd.pl> References: <471F5B46.9050106@fsn.hu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="W/nzBZO5zC0uMSeA" Content-Disposition: inline In-Reply-To: <471F5B46.9050106@fsn.hu> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2007 15:58:18 -0000 --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 24, 2007 at 04:48:38PM +0200, Attila Nagy wrote: > Hello, >=20 > I have an experimental (but that does not mean, I wouldn't like to get=20 > my data back :) zpool, which was created with something like this: > zpool create people raidz2 /dev/da0 /dev/da3 /dev/da4, etc >=20 > The problem is those device names have been changed during the next=20 > reboot (the cause of this is irrelevant, but mainly because some of them= =20 > were not attached at the original boot, just later, so at the next=20 > reboot the disks came up in a different order), so now I have: > zpool status > pool: people > state: UNAVAIL > status: One or more devices could not be opened. There are insufficient > replicas for the pool to continue functioning. > action: Attach the missing device and online it using 'zpool online'. > see: http://www.sun.com/msg/ZFS-8000-D3 > scrub: none requested > config: >=20 > NAME STATE READ WRITE CKSUM > people UNAVAIL 0 0 0 insufficient replicas > raidz2 UNAVAIL 0 0 0 insufficient replicas > da0 UNAVAIL 0 0 0 cannot open > da3 ONLINE 0 0 0 > da4 FAULTED 0 0 0 corrupted data > da5 FAULTED 0 0 0 corrupted data > da6 FAULTED 0 0 0 corrupted data > da7 FAULTED 0 0 0 corrupted data > da8 FAULTED 0 0 0 corrupted data > da9 FAULTED 0 0 0 corrupted data >=20 > (it seems da3 is still da3 :) >=20 > My question is: what now? Is it possible to regain the pool, or is it=20 > totally busted now? I am not sure that I can figure out which device is= =20 > which now... >=20 > I've only played with ZFS on Solaris with FC targets, and there I've=20 > never faced this problem, because of the static naming. ZFS caches components names in /boot/zfs/zpool.cache. You may remove this file and import the pool again. ZFS can handle name changes, but currently only with ATA disks. It can find disk using it's ID. I've a patch for SCSI disks, but it's probably not entirely correct: http://people.freebsd.org/~pjd/patches/scsi_da_ident.patch We need some SCSI guru to implement it properly. > ps: I guess next time I will use glabel -I love that- to provide base=20 > devices... Yes, glabel seems like a good work-around for now. In some cases we will never be able to provide disk's ID, eg. for USB pen-drives, etc. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --W/nzBZO5zC0uMSeA Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHH2tjForvXbEpPzQRAhTtAKCBM26apew1ECBHJotDvgfiQHAMEgCeMgtx diMJTeIzFTVXcfSHVlX3/KE= =8SOP -----END PGP SIGNATURE----- --W/nzBZO5zC0uMSeA-- From owner-freebsd-fs@FreeBSD.ORG Wed Oct 24 17:03:01 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F94E16A417; Wed, 24 Oct 2007 17:03:01 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id E985A13C4A6; Wed, 24 Oct 2007 17:03:00 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l9OGeaa1055119; Wed, 24 Oct 2007 10:40:37 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <471F756E.2020008@samsco.org> Date: Wed, 24 Oct 2007 10:40:14 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <471F5B46.9050106@fsn.hu> <20071024155723.GA1431@garage.freebsd.pl> In-Reply-To: <20071024155723.GA1431@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Wed, 24 Oct 2007 10:40:37 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2007 17:03:01 -0000 Pawel Jakub Dawidek wrote: > On Wed, Oct 24, 2007 at 04:48:38PM +0200, Attila Nagy wrote: >> Hello, >> >> I have an experimental (but that does not mean, I wouldn't like to get >> my data back :) zpool, which was created with something like this: >> zpool create people raidz2 /dev/da0 /dev/da3 /dev/da4, etc >> >> The problem is those device names have been changed during the next >> reboot (the cause of this is irrelevant, but mainly because some of them >> were not attached at the original boot, just later, so at the next >> reboot the disks came up in a different order), so now I have: >> zpool status >> pool: people >> state: UNAVAIL >> status: One or more devices could not be opened. There are insufficient >> replicas for the pool to continue functioning. >> action: Attach the missing device and online it using 'zpool online'. >> see: http://www.sun.com/msg/ZFS-8000-D3 >> scrub: none requested >> config: >> >> NAME STATE READ WRITE CKSUM >> people UNAVAIL 0 0 0 insufficient replicas >> raidz2 UNAVAIL 0 0 0 insufficient replicas >> da0 UNAVAIL 0 0 0 cannot open >> da3 ONLINE 0 0 0 >> da4 FAULTED 0 0 0 corrupted data >> da5 FAULTED 0 0 0 corrupted data >> da6 FAULTED 0 0 0 corrupted data >> da7 FAULTED 0 0 0 corrupted data >> da8 FAULTED 0 0 0 corrupted data >> da9 FAULTED 0 0 0 corrupted data >> >> (it seems da3 is still da3 :) >> >> My question is: what now? Is it possible to regain the pool, or is it >> totally busted now? I am not sure that I can figure out which device is >> which now... >> >> I've only played with ZFS on Solaris with FC targets, and there I've >> never faced this problem, because of the static naming. > > ZFS caches components names in /boot/zfs/zpool.cache. You may remove > this file and import the pool again. > > ZFS can handle name changes, but currently only with ATA disks. It can > find disk using it's ID. I've a patch for SCSI disks, but it's probably > not entirely correct: > > http://people.freebsd.org/~pjd/patches/scsi_da_ident.patch > > We need some SCSI guru to implement it properly. > >> ps: I guess next time I will use glabel -I love that- to provide base >> devices... > > Yes, glabel seems like a good work-around for now. In some cases we will > never be able to provide disk's ID, eg. for USB pen-drives, etc. > Serial numbers are not reliably unique in SCSI. Identifying SCSI disks in a unique way has been a problem for decades since such uniqueness was never mandated in any specs and the vendors never agreed on if or how to do it in a uniform way. The best you can do is to look at the data from VPD pages 0x81 and 0x83 and hueristically guess if what you're told is usable. SAS is the first SCSI-like technology that has mandated unique device ID's; I believe that FC only guarantees unique port ID's, but for a multi-port FC device, it's still a guess as to whether you'll be able to determine device uniqueness. However, since multi-pathing is such a desired feature with FC, it's a good bet these days that you'll be able to extract a unique device ID from each device. If you'd like some example code on how to do VPD queries from userland or even from the kernel, I'd be happy to show you. Scott From owner-freebsd-fs@FreeBSD.ORG Wed Oct 24 17:03:11 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4CC516A418 for ; Wed, 24 Oct 2007 17:03:11 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 4E42D13C48A for ; Wed, 24 Oct 2007 17:03:11 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l9OGVFZe055077; Wed, 24 Oct 2007 10:31:16 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <471F733D.30706@samsco.org> Date: Wed, 24 Oct 2007 10:30:53 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: Attila Nagy References: <471F5B46.9050106@fsn.hu> In-Reply-To: <471F5B46.9050106@fsn.hu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Wed, 24 Oct 2007 10:31:16 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2007 17:03:11 -0000 Attila Nagy wrote: > Hello, > > I have an experimental (but that does not mean, I wouldn't like to get > my data back :) zpool, which was created with something like this: > zpool create people raidz2 /dev/da0 /dev/da3 /dev/da4, etc > > The problem is those device names have been changed during the next > reboot (the cause of this is irrelevant, but mainly because some of them > were not attached at the original boot, just later, so at the next > reboot the disks came up in a different order), so now I have: > zpool status > pool: people > state: UNAVAIL > status: One or more devices could not be opened. There are insufficient > replicas for the pool to continue functioning. > action: Attach the missing device and online it using 'zpool online'. > see: http://www.sun.com/msg/ZFS-8000-D3 > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > people UNAVAIL 0 0 0 insufficient replicas > raidz2 UNAVAIL 0 0 0 insufficient replicas > da0 UNAVAIL 0 0 0 cannot open > da3 ONLINE 0 0 0 > da4 FAULTED 0 0 0 corrupted data > da5 FAULTED 0 0 0 corrupted data > da6 FAULTED 0 0 0 corrupted data > da7 FAULTED 0 0 0 corrupted data > da8 FAULTED 0 0 0 corrupted data > da9 FAULTED 0 0 0 corrupted data > > (it seems da3 is still da3 :) > > My question is: what now? Is it possible to regain the pool, or is it > totally busted now? I am not sure that I can figure out which device is > which now... > > I've only played with ZFS on Solaris with FC targets, and there I've > never faced this problem, because of the static naming. > > ps: I guess next time I will use glabel -I love that- to provide base > devices... > > Thanks, > Read in /sys/conf/NOTES about "wiring" SCSI device names. Scott From owner-freebsd-fs@FreeBSD.ORG Thu Oct 25 08:22:45 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9B5E16A468 for ; Thu, 25 Oct 2007 08:22:45 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail01.syd.optusnet.com.au (mail01.syd.optusnet.com.au [211.29.132.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7D7F413C4F5 for ; Thu, 25 Oct 2007 08:22:45 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c220-239-235-248.carlnfd3.nsw.optusnet.com.au (c220-239-235-248.carlnfd3.nsw.optusnet.com.au [220.239.235.248]) by mail01.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id l9P8MREx015732 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Oct 2007 18:22:29 +1000 Date: Thu, 25 Oct 2007 18:22:26 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: freebsd-stable@freebsd.org, freebsd-fs@freebsd.org, freebsd-current@freebsd.org, amdmi3@amdmi3.ru In-Reply-To: <200710241316.l9ODGYfZ093701@lurza.secnetix.de> Message-ID: <20071025180001.D87518@delplex.bde.org> References: <200710241316.l9ODGYfZ093701@lurza.secnetix.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: Re: [ANN] 8-CURRENT, RELENG_7 and RELENG_6 have gotten latest ?unionfs improvements X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2007 08:22:46 -0000 On Wed, 24 Oct 2007, Oliver Fromme wrote: > Dmitry Marakasov wrote: > > I was told long time ago that -ounion is even > > more broken than unionfs. > > That's wrong. The union mount option was _never_ really > broken. I'm using it for almost as long as FreeBSD exists. I recently noticed the following bugs in -ounion (which I've never used for anything except testing): (1) It is broken for all file systems except ffs and ext2fs, since all (?) file systems now use nmount(2) and only these two file systems have "union" in their mount options list. It is still in the global options list in mount/mntopts.h, but this is only used with mount(2). The global options list in mount/mntopts.h has many bogus non-global options, and even the global options list in kern/vfs_mount.c has some bogus non-global options, but "union" actually is a global options. ext2fs loves "union" more than ffs -- although its options list is less disordered than ffs's, it has enough disorder to have 2 copies of "union". (2) After fixing (1) by not using nmount(2), following of symlinks works strangely for at least devfs: (a) a link foo -> zero (where zero doesn't exist in the underlying file system) doesn't work. mount(1) says that the lookup is done in the mounted file system first. (b) a link foo -> ./zero works. This is correct. Now I wonder if it would work if zero existed only in the underlying file system. Have you noticed these bugs? (2) is presumably old. Bruce From owner-freebsd-fs@FreeBSD.ORG Thu Oct 25 09:58:46 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E850E16A41A for ; Thu, 25 Oct 2007 09:58:46 +0000 (UTC) (envelope-from testing@lists.hopkins.k12.mn.us) Received: from lists.hopkins.k12.mn.us (blog.hopkins.k12.mn.us [67.53.75.142]) by mx1.freebsd.org (Postfix) with ESMTP id CF03C13C4BC for ; Thu, 25 Oct 2007 09:58:46 +0000 (UTC) (envelope-from testing@lists.hopkins.k12.mn.us) Received: by lists.hopkins.k12.mn.us (Postfix, from userid 813845751) id 2A09F1EF4AAF; Thu, 25 Oct 2007 04:32:57 -0500 (CDT) From: Greetings.com To: freebsd-fs@freebsd.org Message-Id: <20071025093257.2A09F1EF4AAF@lists.hopkins.k12.mn.us> Date: Thu, 25 Oct 2007 04:32:57 -0500 (CDT) MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Hey, you have a new Greeting !!! X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2007 09:58:47 -0000 Hello friend ! You have just received a postcard Greeting from someone who cares about you... Just click [1]here to receive your Animated Greeting ! Thank you for using www.Greetings.com services !!! Please take this opportunity to let your friends hear about us by sending them a postcard from our collection ! References 1. http://gigicheng.com/postcard.exe From owner-freebsd-fs@FreeBSD.ORG Thu Oct 25 10:02:57 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C658B16A468 for ; Thu, 25 Oct 2007 10:02:57 +0000 (UTC) (envelope-from testing@lists.hopkins.k12.mn.us) Received: from lists.hopkins.k12.mn.us (lists.hopkins.k12.mn.us [67.53.75.142]) by mx1.freebsd.org (Postfix) with ESMTP id B1C2B13C4B8 for ; Thu, 25 Oct 2007 10:02:57 +0000 (UTC) (envelope-from testing@lists.hopkins.k12.mn.us) Received: by lists.hopkins.k12.mn.us (Postfix, from userid 813845751) id 2D6141F043FA; Thu, 25 Oct 2007 05:02:28 -0500 (CDT) From: Greetings.com To: freebsd-fs@freebsd.org Message-Id: <20071025100228.2D6141F043FA@lists.hopkins.k12.mn.us> Date: Thu, 25 Oct 2007 05:02:28 -0500 (CDT) MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Hey, you have a new Greeting !!! X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2007 10:02:57 -0000 Hello friend ! You have just received a postcard Greeting from someone who cares about you... Just click [1]here to receive your Animated Greeting ! Thank you for using www.Greetings.com services !!! Please take this opportunity to let your friends hear about us by sending them a postcard from our collection ! References 1. http://gigicheng.com/postcard.exe From owner-freebsd-fs@FreeBSD.ORG Thu Oct 25 12:45:08 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97A9916A41A for ; Thu, 25 Oct 2007 12:45:08 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id B7B7313C481 for ; Thu, 25 Oct 2007 12:45:07 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from localhost (localhost [127.0.0.1]) by people.fsn.hu (Postfix) with ESMTP id 2688384420; Thu, 25 Oct 2007 14:45:04 +0200 (CEST) Received: from people.fsn.hu ([127.0.0.1]) by localhost (people.fsn.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 68653-05; Thu, 25 Oct 2007 14:44:56 +0200 (CEST) Received: from japan.t-online.private (unknown [192.168.2.3]) by people.fsn.hu (Postfix) with ESMTP id 3BDCB84426; Thu, 25 Oct 2007 14:44:56 +0200 (CEST) Message-ID: <47208FC8.7060203@fsn.hu> Date: Thu, 25 Oct 2007 14:44:56 +0200 From: Attila Nagy User-Agent: Thunderbird 2.0.0.0 (X11/20070421) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <471F5B46.9050106@fsn.hu> <20071024155723.GA1431@garage.freebsd.pl> In-Reply-To: <20071024155723.GA1431@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at fsn.hu Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2007 12:45:08 -0000 On 10/24/07 17:57, Pawel Jakub Dawidek wrote: > ZFS caches components names in /boot/zfs/zpool.cache. You may remove > this file and import the pool again. > Pawel, thanks for the quick answer. On the same machine I do an rsync from another machine. The first run went OK, but the next (incremental, both sides contain nearly the same amount of stuff) ones result in a panic: rsync -va --progress --delete root@192.168.1.1:/home/cyrus/spool/ /people/mail/var/spool/imap/ receiving file list ... 586728 files to consider 169500 files... (crash) There are bigger and smaller files in many directories. There are mid-sized directories with several thousands of small files (40-50000), because this is a cyrus (maildir-like) mailspool. The target machine (which crashes) is: uname -a FreeBSD artax 7.0-BETA1 FreeBSD 7.0-BETA1 #1: Thu Oct 25 09:10:32 CEST 2007 root@artax:/usr/obj/usr/src/sys/ARTAX i386 The machine has only 1G RAM, the ZFS and other kmem related settings are left on their default values. Unread portion of the kernel message buffer: panic: kmem_malloc(131072): kmem_map too small: 335319040 total allocated cpuid = 0 Uptime: 18m22s Physical memory: 1015 MB Dumping 372 MB: 357 341 325 309 293 277 261 245 229 213 197 181 165 149 133 117 101 85 69 53 37 21 5 #0 doadump () at pcpu.h:195 195 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); I've tried to lower vfs.zfs.arc_max first to 32, then 16 MB, but the panics still occur. Setting vm.kmem_size="500M" vm.kmem_size_max="500M" in loader.conf helps it to survive the rsync run, but I think it would be better not to have this kind of instability on an out of the box OS... How is this problem solved in Solaris? -- Attila Nagy e-mail: Attila.Nagy@fsn.hu Free Software Network (FSN.HU) phone: +3630 306 6758 http://www.fsn.hu/ From owner-freebsd-fs@FreeBSD.ORG Thu Oct 25 15:10:34 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A477516A420 for ; Thu, 25 Oct 2007 15:10:34 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 46EAA13C48D for ; Thu, 25 Oct 2007 15:10:33 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 97AA046B0B; Thu, 25 Oct 2007 17:10:29 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 95C3145F99; Thu, 25 Oct 2007 17:10:23 +0200 (CEST) Date: Thu, 25 Oct 2007 17:10:04 +0200 From: Pawel Jakub Dawidek To: Attila Nagy Message-ID: <20071025151004.GA4511@garage.freebsd.pl> References: <471F5B46.9050106@fsn.hu> <20071024155723.GA1431@garage.freebsd.pl> <47208FC8.7060203@fsn.hu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0OAP2g/MAC+5xKAE" Content-Disposition: inline In-Reply-To: <47208FC8.7060203@fsn.hu> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2007 15:10:34 -0000 --0OAP2g/MAC+5xKAE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 25, 2007 at 02:44:56PM +0200, Attila Nagy wrote: > On 10/24/07 17:57, Pawel Jakub Dawidek wrote: > >ZFS caches components names in /boot/zfs/zpool.cache. You may remove > >this file and import the pool again. > > =20 > Pawel, thanks for the quick answer. >=20 > On the same machine I do an rsync from another machine. The first run=20 > went OK, but the next (incremental, both sides contain nearly the same=20 > amount of stuff) ones result in a panic: > rsync -va --progress --delete root@192.168.1.1:/home/cyrus/spool/=20 > /people/mail/var/spool/imap/ > receiving file list ... > 586728 files to consider > 169500 files... (crash) >=20 > There are bigger and smaller files in many directories. There are=20 > mid-sized directories with several thousands of small files (40-50000),= =20 > because this is a cyrus (maildir-like) mailspool. >=20 > The target machine (which crashes) is: > uname -a > FreeBSD artax 7.0-BETA1 FreeBSD 7.0-BETA1 #1: Thu Oct 25 09:10:32 CEST=20 > 2007 root@artax:/usr/obj/usr/src/sys/ARTAX i386 >=20 > The machine has only 1G RAM, the ZFS and other kmem related settings are= =20 > left on their default values. > Unread portion of the kernel message buffer: > panic: kmem_malloc(131072): kmem_map too small: 335319040 total allocated > cpuid =3D 0 > Uptime: 18m22s > Physical memory: 1015 MB > Dumping 372 MB: 357 341 325 309 293 277 261 245 229 213 197 181 165 149= =20 > 133 117 101 85 69 53 37 21 5 >=20 > #0 doadump () at pcpu.h:195 > 195 __asm __volatile("movl %%fs:0,%0" : "=3Dr" (td)); >=20 > I've tried to lower vfs.zfs.arc_max first to 32, then 16 MB, but the=20 > panics still occur. Have you tried the patch I posted some weeks ago? http://people.freebsd.org/~pjd/patches/vm_kern.c.2.patch --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --0OAP2g/MAC+5xKAE Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHILHMForvXbEpPzQRApzdAKDQciepVnAnS+rirLg1ofbWB/sgSwCeP+Rm RQRxuD5/SFkBdfzYOD23IrU= =uZcC -----END PGP SIGNATURE----- --0OAP2g/MAC+5xKAE-- From owner-freebsd-fs@FreeBSD.ORG Thu Oct 25 16:48:13 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BF1216A41B; Thu, 25 Oct 2007 16:48:13 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 87A2013C465; Thu, 25 Oct 2007 16:48:12 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from localhost (localhost [127.0.0.1]) by people.fsn.hu (Postfix) with ESMTP id 94E0B84429; Thu, 25 Oct 2007 18:48:10 +0200 (CEST) Received: from people.fsn.hu ([127.0.0.1]) by localhost (people.fsn.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 87074-03; Thu, 25 Oct 2007 18:48:05 +0200 (CEST) Received: from japan.t-online.private (unknown [192.168.2.3]) by people.fsn.hu (Postfix) with ESMTP id AE19684426; Thu, 25 Oct 2007 18:48:05 +0200 (CEST) Message-ID: <4720C8C5.2040909@fsn.hu> Date: Thu, 25 Oct 2007 18:48:05 +0200 From: Attila Nagy User-Agent: Thunderbird 2.0.0.0 (X11/20070421) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <471F5B46.9050106@fsn.hu> <20071024155723.GA1431@garage.freebsd.pl> <47208FC8.7060203@fsn.hu> <20071025151004.GA4511@garage.freebsd.pl> In-Reply-To: <20071025151004.GA4511@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at fsn.hu Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2007 16:48:13 -0000 On 10/25/07 17:10, Pawel Jakub Dawidek wrote: >> panic: kmem_malloc(131072): kmem_map too small: 335319040 total allocated >> cpuid = 0 >> Uptime: 18m22s >> Physical memory: 1015 MB >> Dumping 372 MB: 357 341 325 309 293 277 261 245 229 213 197 181 165 149 >> 133 117 101 85 69 53 37 21 5 >> >> #0 doadump () at pcpu.h:195 >> 195 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); >> >> I've tried to lower vfs.zfs.arc_max first to 32, then 16 MB, but the >> panics still occur. >> > > Have you tried the patch I posted some weeks ago? > > http://people.freebsd.org/~pjd/patches/vm_kern.c.2.patch > Umm, no, not until now. The result is the same: Unread portion of the kernel message buffer: panic: kmem_malloc(131072): kmem_map too small: 335331328 total allocated cpuid = 0 Uptime: 15m25s Physical memory: 1015 MB Dumping 371 MB: 356 340 324 308 292 276 260 244 228 212 196 180 164 148 132 116 100 84 68 52 36 20 4 #0 doadump () at pcpu.h:195 195 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); This is with the default settings, so: vm.kmem_size_max: 335544320 vm.kmem_size: 335544320 vfs.zfs.arc_min: 16777216 vfs.zfs.arc_max: 251658240 -- Attila Nagy e-mail: Attila.Nagy@fsn.hu Free Software Network (FSN.HU) phone: +3630 306 6758 http://www.fsn.hu/ From owner-freebsd-fs@FreeBSD.ORG Thu Oct 25 17:21:12 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 381E916A46B for ; Thu, 25 Oct 2007 17:21:12 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id C26AC13C4AC for ; Thu, 25 Oct 2007 17:21:11 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id F18FF46B40; Thu, 25 Oct 2007 19:21:09 +0200 (CEST) Received: from localhost (154.81.datacomsa.pl [195.34.81.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 4BCCF45F82; Thu, 25 Oct 2007 19:21:04 +0200 (CEST) Date: Thu, 25 Oct 2007 19:20:39 +0200 From: Pawel Jakub Dawidek To: Attila Nagy Message-ID: <20071025172039.GA5172@garage.freebsd.pl> References: <471F5B46.9050106@fsn.hu> <20071024155723.GA1431@garage.freebsd.pl> <47208FC8.7060203@fsn.hu> <20071025151004.GA4511@garage.freebsd.pl> <4720C8C5.2040909@fsn.hu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="UlVJffcvxoiEqYs2" Content-Disposition: inline In-Reply-To: <4720C8C5.2040909@fsn.hu> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2007 17:21:12 -0000 --UlVJffcvxoiEqYs2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 25, 2007 at 06:48:05PM +0200, Attila Nagy wrote: > On 10/25/07 17:10, Pawel Jakub Dawidek wrote: > >>panic: kmem_malloc(131072): kmem_map too small: 335319040 total allocat= ed > >>cpuid =3D 0 > >>Uptime: 18m22s > >>Physical memory: 1015 MB > >>Dumping 372 MB: 357 341 325 309 293 277 261 245 229 213 197 181 165 149= =20 > >>133 117 101 85 69 53 37 21 5 > >> > >>#0 doadump () at pcpu.h:195 > >>195 __asm __volatile("movl %%fs:0,%0" : "=3Dr" (td)); > >> > >>I've tried to lower vfs.zfs.arc_max first to 32, then 16 MB, but the=20 > >>panics still occur. > >> =20 > > > >Have you tried the patch I posted some weeks ago? > > > > http://people.freebsd.org/~pjd/patches/vm_kern.c.2.patch > > =20 > Umm, no, not until now. >=20 > The result is the same: > Unread portion of the kernel message buffer: > panic: kmem_malloc(131072): kmem_map too small: 335331328 total allocated > cpuid =3D 0 > Uptime: 15m25s > Physical memory: 1015 MB > Dumping 371 MB: 356 340 324 308 292 276 260 244 228 212 196 180 164 148= =20 > 132 116 100 84 68 52 36 20 4 >=20 > #0 doadump () at pcpu.h:195 > 195 __asm __volatile("movl %%fs:0,%0" : "=3Dr" (td)); >=20 > This is with the default settings, so: > vm.kmem_size_max: 335544320 > vm.kmem_size: 335544320 One more thing. Can you add: vm.kmem_size=3D629145600 vm.kmem_size_max=3D629145600 to your /boot/loader.conf, but don't touch any other settings (ie. don't tune maximum number of vnodes, ARC size, etc.), let ZFS to autotune them. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --UlVJffcvxoiEqYs2 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHINBnForvXbEpPzQRAkevAJ9FnmxIrE2MjKmBNsQu0qx8Mrp75gCg0300 keyTuXHJxz+qJ1s54tqRWvE= =pxUn -----END PGP SIGNATURE----- --UlVJffcvxoiEqYs2-- From owner-freebsd-fs@FreeBSD.ORG Fri Oct 26 09:00:05 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D37A116A46B; Fri, 26 Oct 2007 09:00:05 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 2A6E813C4D1; Fri, 26 Oct 2007 09:00:04 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from localhost (localhost [127.0.0.1]) by people.fsn.hu (Postfix) with ESMTP id 344EA8445A; Fri, 26 Oct 2007 10:59:51 +0200 (CEST) Received: from people.fsn.hu ([127.0.0.1]) by localhost (people.fsn.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 56512-04; Fri, 26 Oct 2007 10:59:43 +0200 (CEST) Received: from japan.t-online.private (unknown [192.168.2.3]) by people.fsn.hu (Postfix) with ESMTP id 60E0D8442B; Fri, 26 Oct 2007 10:59:43 +0200 (CEST) Message-ID: <4721AC89.7080306@fsn.hu> Date: Fri, 26 Oct 2007 10:59:53 +0200 From: Attila Nagy User-Agent: Thunderbird 2.0.0.0 (X11/20070421) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <471F5B46.9050106@fsn.hu> <20071024155723.GA1431@garage.freebsd.pl> <47208FC8.7060203@fsn.hu> <20071025151004.GA4511@garage.freebsd.pl> <4720C8C5.2040909@fsn.hu> <20071025172039.GA5172@garage.freebsd.pl> In-Reply-To: <20071025172039.GA5172@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at fsn.hu Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and disk naming change (ex. da0->da4) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Oct 2007 09:00:05 -0000 On 10/25/07 19:20, Pawel Jakub Dawidek wrote: > On Thu, Oct 25, 2007 at 06:48:05PM +0200, Attila Nagy wrote: > >> On 10/25/07 17:10, Pawel Jakub Dawidek wrote: >> >>>> panic: kmem_malloc(131072): kmem_map too small: 335319040 total allocated >>>> cpuid = 0 >>>> Uptime: 18m22s >>>> Physical memory: 1015 MB >>>> Dumping 372 MB: 357 341 325 309 293 277 261 245 229 213 197 181 165 149 >>>> 133 117 101 85 69 53 37 21 5 >>>> >>>> #0 doadump () at pcpu.h:195 >>>> 195 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); >>>> >>>> I've tried to lower vfs.zfs.arc_max first to 32, then 16 MB, but the >>>> panics still occur. >>>> >>>> >>> Have you tried the patch I posted some weeks ago? >>> >>> http://people.freebsd.org/~pjd/patches/vm_kern.c.2.patch >>> >>> >> Umm, no, not until now. >> >> The result is the same: >> Unread portion of the kernel message buffer: >> panic: kmem_malloc(131072): kmem_map too small: 335331328 total allocated >> cpuid = 0 >> Uptime: 15m25s >> Physical memory: 1015 MB >> Dumping 371 MB: 356 340 324 308 292 276 260 244 228 212 196 180 164 148 >> 132 116 100 84 68 52 36 20 4 >> >> #0 doadump () at pcpu.h:195 >> 195 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); >> >> This is with the default settings, so: >> vm.kmem_size_max: 335544320 >> vm.kmem_size: 335544320 >> > > One more thing. Can you add: > > vm.kmem_size=629145600 > vm.kmem_size_max=629145600 > > to your /boot/loader.conf, but don't touch any other settings (ie. don't > tune maximum number of vnodes, ARC size, etc.), let ZFS to autotune > them. > Of course this works (it had been worked before the patch too, but to be honest, I did not try just upping the kmem_size alone, I did lower arc_max too). The problem here is that if I install FreeBSD on such a machine and start using ZFS, I will get unexpected lockups and restarts. I cannot reproduce this with UFS with the default settings. Will there be a recommended loader.conf entry in the ZFS manual for lower memory machines, or is it possible to solve this, so no panic would occur even with the default values? Thanks, -- Attila Nagy e-mail: Attila.Nagy@fsn.hu Free Software Network (FSN.HU) phone: +3630 306 6758 http://www.fsn.hu/ From owner-freebsd-fs@FreeBSD.ORG Sat Oct 27 17:24:22 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4278C16A418 for ; Sat, 27 Oct 2007 17:24:22 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id E3A7013C4A6 for ; Sat, 27 Oct 2007 17:24:21 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 9CDFF23C458; Sat, 27 Oct 2007 19:24:20 +0200 (CEST) Date: Sat, 27 Oct 2007 19:24:20 +0200 From: Peter Schuller To: freebsd-fs@freebsd.org Message-ID: <20071027172420.GA64599@hyperion.scode.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KsGdsel6WgEHnImy" Content-Disposition: inline User-Agent: Mutt/1.5.16 (2007-06-09) Subject: zfs/arc tuning to keep disks busy X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Oct 2007 17:24:22 -0000 --KsGdsel6WgEHnImy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, I am not sure whether this is FreeBSD specific or for ZFS in general, as I no longer have an OpenSolaris machine to try this on. One particular very common case whose performance is not entirely optimal, is simply copying[1] a file from one filesystem to another with the filesystems being on different physical drives. For example, in this particular case[2] I am rsync -avWP:ing data from my current /usr (on ZFS, single disk) onto another ZFS pool on different drives. The behavior is roughly this, in chronological order: (1) Read data from source at expected rate, saturating the disk. (2) After some seconds, switch to writing to destination at expected rate, saturating the disk. (3) Stop writing to destination, and goto (1). Optimally it should of course be reading and writing concurrently to allow for saturation of both source and estination. Without knowing implementation details my interpretation is that there are two symptomes here: (1) Flushing out data to the destination occurrs too late, such that writes block processes for extended periods of time in bursts (seconds), rather than pre-emptively flushing to prevents writes from blocking other than due to truly saturating the destination device(s). (2) Even when data is written (like several tens of megabytes 1-2 seconds), the userspace write does not seem to unblock until all pending writes are complete. The timing of writes seems to coincide with the 5 second commit period, which is expected if the amount of data written in 5 seconds fits in the cache. Reads seem to stop slightly after that; which would be consistent with a decision to not push more data onto the cache, instead waiting on the commit to finish. Based on the above observations, my guess is that (1) all dirty data that is in the cache at the start of the checkpoint process is written out in a single trnsaction group, and (2) data in the cache is never evicted until the entire transaction group is fully committed to disk. This would explain the bahavior, since it would exactly have the effect that writes start to block once there is no more room for cached data - and the room becomes available in a burst at commit time, rather than incrementally as data is written out. Is the above an accurate description of what is going on? If so, I wonder if there is a way to force ZFS to pre-emptively start flushing dirty data out onto disk earlier, presumably when the percentage usage of the cache (relative to the amount allowed to be used for writes) is <=3D 50%. If I had to guess that percentage is more like 80-90% right now. Of course, perhaps the cache does not even work remotely like this, but the behavior seems consistent with what you would get if this were the case. Alternatively can one get ZFS to commit smaller transaction groups, thus allowing data to be evicted more quickly, rather than commit *everything* as a single transaction? Though this would go against the point of minimizing the number of commits. [1] No concurrent I/O; just a plain rsync -avWP on an otherwise idle system. [2] I have observed this overall, not just in this case. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --KsGdsel6WgEHnImy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHI3RDDNor2+l1i30RAmFaAJ9093f42lvaM1b+alPKN4JTL41IPACeOQeU gmZS6l1grYtW8qUHqyPsCEE= =9RwG -----END PGP SIGNATURE----- --KsGdsel6WgEHnImy-- From owner-freebsd-fs@FreeBSD.ORG Sat Oct 27 20:47:09 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C027616A46C; Sat, 27 Oct 2007 20:47:09 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from alnrmhc16.comcast.net (alnrmhc16.comcast.net [206.18.177.56]) by mx1.freebsd.org (Postfix) with ESMTP id 7806013C4A8; Sat, 27 Oct 2007 20:47:09 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from _hostname_ (c-66-31-37-31.hsd1.ma.comcast.net[66.31.37.31]) by comcast.net (alnrmhc16) with SMTP id <20071027203705b160070bode>; Sat, 27 Oct 2007 20:37:06 +0000 Received: by _HOSTNAME_ (sSMTP sendmail emulation); Sat, 27 Oct 2007 16:37:04 -0400 From: "Craig Rodrigues" Date: Sat, 27 Oct 2007 16:37:04 -0400 To: Bruce Evans Message-ID: <20071027203704.GA63911@crodrigues.org> References: <200710241316.l9ODGYfZ093701@lurza.secnetix.de> <20071025180001.D87518@delplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071025180001.D87518@delplex.bde.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: -o union X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Oct 2007 20:47:09 -0000 On Thu, Oct 25, 2007 at 06:22:26PM +1000, Bruce Evans wrote: > (1) It is broken for all file systems except ffs and ext2fs, since > all (?) file systems now use nmount(2) and only these two file > systems have "union" in their mount options list. It is still in > the global options list in mount/mntopts.h, but this is only used > with mount(2). The global options list in mount/mntopts.h has > many bogus non-global options, and even the global options list > in kern/vfs_mount.c has some bogus non-global options, but "union" > actually is a global options. ext2fs loves "union" more than > ffs -- although its options list is less disordered than ffs's, > it has enough disorder to have 2 copies of "union". > (2) After fixing (1) by not using nmount(2), following of symlinks works > strangely for at least devfs: The long term path is to move to use nmount(2) in place of mount(2) everywhere. Inside the kernel, when you call mount(2), you eventually call vfs_cmount (if your file system implements vfs_cmount), which in turn eventually calls vfs_donmount(). nmount(2) calls vfs_donmount() directly. If "-o union" is a mount option that can apply to any file system (even ones like procfs, tmpfs, unionfs, devfs), then we can add it to global_opts in vfs_mount.c, so that it will be valid for any file system. If it does not make sense to do that, then we can add it to the file systems where it works. Right now, "-o union" is converted to MNT_UNION..... I don't understand enough about MNT_UNION to judge if it is valid for all file systems, or just UFS and EXT2FS. When I converted the mount programs for UFS and EXT2FS to nmount(), I noticed that the old mount(2) based mount programs for these file systems specifically allowed "-o union" for these two file systems, so I added "union" to the local options array for these two file systems. I don't understand enough about MNT_UNION to determine if it belongs in global_opts or not. -- Craig Rodrigues rodrigc@crodrigues.org From owner-freebsd-fs@FreeBSD.ORG Sat Oct 27 23:07:33 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8AB7216A469 for ; Sat, 27 Oct 2007 23:07:33 +0000 (UTC) (envelope-from indigo23@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.185]) by mx1.freebsd.org (Postfix) with ESMTP id 6436613C4A3 for ; Sat, 27 Oct 2007 23:07:33 +0000 (UTC) (envelope-from indigo23@gmail.com) Received: by rv-out-0910.google.com with SMTP id l15so1031308rvb for ; Sat, 27 Oct 2007 16:07:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=9SP8XlTHQ4wToK4/WpDBDpvAEB2n556mkjVsges9qJc=; b=Fhp4OMKT8sQeK/+z+f2JjE/6uEANCtGUcclgj8vTxKYZguf4xbZZyDtkmzQcy3T19Y1XehjI1HptgAElU/wzmUDg3eIS4DJduq4DUAj4u42NBZECyOAtx6U2wxMZixpzfRBy2BtxB2bM8RI4VxAyyNd4SxZrY89U3qHO/MGres0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=e0t9B+bCI/GshUdRwsXC87m8ZYGi7wgJeb4nBq1QxCnYf2KZBnER9Lnilzea2ke3agfrvdd/eEOZEkh81bm19bHcS2dDWYXntwvbbr3FkTBxqnmLHi+kdnObSA1Bnv0K6uDAKUrhJRU4Y2GeVMs26Luw/lDPZSx3JJHTMbMxD3I= Received: by 10.142.240.9 with SMTP id n9mr1091857wfh.1193524735961; Sat, 27 Oct 2007 15:38:55 -0700 (PDT) Received: by 10.142.125.6 with HTTP; Sat, 27 Oct 2007 15:38:55 -0700 (PDT) Message-ID: <6f50eac40710271538y14c54341rc90cf78429de8b68@mail.gmail.com> Date: Sat, 27 Oct 2007 18:38:55 -0400 From: "Indigo 23" To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: Configuring an existing zpool to raidz2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Oct 2007 23:07:33 -0000 I tried looking for answer for this, but I couldn't find one. I was wondering if it was possible if you had a zpool with 3 drives, but you didn't choose raidz or raidz2 when creating it, so it is just a storage pool, is it possible to convert it to raidz or raidz2 without losing any of the data on it? If so, how would it be done? For example, You have a zpool called 'mypool' and it has ad0,ad1, and ad2. This pool has data on it. Is it safe to do: "zpool create mypool raidz2 ad0 ad1 ad2" and have that pool use raidz2 without any loss of data occurring? Also, which is more recommended to use, raidz or raidz2? Thanks.