From owner-freebsd-fs@FreeBSD.ORG Sun Oct 7 07:55:51 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 81D001065740; Sun, 7 Oct 2012 07:55:51 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 711318FC08; Sun, 7 Oct 2012 07:55:49 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA12998; Sun, 07 Oct 2012 10:55:48 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TKliG-0007sk-7B; Sun, 07 Oct 2012 10:55:48 +0300 Message-ID: <50713582.9040600@FreeBSD.org> Date: Sun, 07 Oct 2012 10:55:46 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: Pawel Jakub Dawidek , "freebsd-fs@freebsd.org" X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: Subject: zfs_remove: delete_now case X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Oct 2012 07:55:51 -0000 It seems that the delete_now path is never taken in zfs_remove(). This is probably good in the bug-cancels-bug way... On FreeBSD VOP_REMOVE is always called with vp being referenced. zfs_remove doesn't take advantage of VOP_REMOVE interface. It ignores the vp argument and instead re-looks up a directory entry id by name (a small performance hit here) and then uses zfs_zget, which adds another reference to the entry's vnode. Thus, a reference count of the vnode is always not less than two. So may_delete_now and delete_now are always false. Why this is good? Because FreeBSD VFS doesn't support direct destruction (or corruption) of the vnode in VOP_REMOVE. It expects to still have a valid vnode with a valid reference count after VOP_REMOVE and then calls vput/vrele on it. But the code in the delete_now branch does some nasty things. It directly decrements the use count and it directly destroys the underlying znode (which is fine in Solaris but not in FreeBSD). But FreeBSD VFS wouldn't even have a chance to panic on the damaged vnode because ZFS code would sooner panic in zfs_znode_delete -> zfs_znode_free -> ASSERT(ZTOV(zp) == NULL) [a FreeBSD-specific assertion). I think that we should make zfs_remove code less confusing and more FreeBSD friendly. We should explicitly rely on zfs_inactive doing the right thing after VOP_REMOVE and drop all the "direct action" code. What do you think? -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Sun Oct 7 09:20:10 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7EEF6106564A; Sun, 7 Oct 2012 09:20:10 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 434BC8FC08; Sun, 7 Oct 2012 09:20:09 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 3AB7A8E5; Sun, 7 Oct 2012 11:19:00 +0200 (CEST) Date: Sun, 7 Oct 2012 11:20:37 +0200 From: Pawel Jakub Dawidek To: Andriy Gapon Message-ID: <20121007092037.GB28611@garage.freebsd.pl> References: <50713582.9040600@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="BwCQnh7xodEAoBMC" Content-Disposition: inline In-Reply-To: <50713582.9040600@FreeBSD.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-fs@freebsd.org" Subject: Re: zfs_remove: delete_now case X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Oct 2012 09:20:10 -0000 --BwCQnh7xodEAoBMC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Oct 07, 2012 at 10:55:46AM +0300, Andriy Gapon wrote: >=20 > It seems that the delete_now path is never taken in zfs_remove(). > This is probably good in the bug-cancels-bug way... >=20 > On FreeBSD VOP_REMOVE is always called with vp being referenced. zfs_rem= ove > doesn't take advantage of VOP_REMOVE interface. It ignores the vp argume= nt and > instead re-looks up a directory entry id by name (a small performance hit= here) > and then uses zfs_zget, which adds another reference to the entry's vnode. > Thus, a reference count of the vnode is always not less than two. So > may_delete_now and delete_now are always false. >=20 > Why this is good? Because FreeBSD VFS doesn't support direct destruction= (or > corruption) of the vnode in VOP_REMOVE. It expects to still have a valid= vnode > with a valid reference count after VOP_REMOVE and then calls vput/vrele o= n it. > But the code in the delete_now branch does some nasty things. It directly > decrements the use count and it directly destroys the underlying znode (w= hich is > fine in Solaris but not in FreeBSD). > But FreeBSD VFS wouldn't even have a chance to panic on the damaged vnode > because ZFS code would sooner panic in zfs_znode_delete -> zfs_znode_free= -> > ASSERT(ZTOV(zp) =3D=3D NULL) [a FreeBSD-specific assertion). >=20 > I think that we should make zfs_remove code less confusing and more FreeB= SD > friendly. We should explicitly rely on zfs_inactive doing the right thin= g after > VOP_REMOVE and drop all the "direct action" code. >=20 > What do you think? I'm fully aware of this code path being dead on FreeBSD. It is left there only to minimize diff against vendor, so I'd prefer not to remove it. Surrounding the code with '#ifdef sun' or similar is ok, I think. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --BwCQnh7xodEAoBMC Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBxSWUACgkQForvXbEpPzT7kgCdF6+4xV+L7aF17HtZFxFVBL+v gjUAniJwvbeAzHRvlHUEBdxjh+6roDiy =UJy4 -----END PGP SIGNATURE----- --BwCQnh7xodEAoBMC-- From owner-freebsd-fs@FreeBSD.ORG Sun Oct 7 09:34:14 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 48A121065670; Sun, 7 Oct 2012 09:34:14 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4E8CD8FC0C; Sun, 7 Oct 2012 09:34:12 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA13438; Sun, 07 Oct 2012 12:34:11 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TKnFT-0007yG-6X; Sun, 07 Oct 2012 12:34:11 +0300 Message-ID: <50714C91.4080407@FreeBSD.org> Date: Sun, 07 Oct 2012 12:34:09 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <50713582.9040600@FreeBSD.org> <20121007092037.GB28611@garage.freebsd.pl> In-Reply-To: <20121007092037.GB28611@garage.freebsd.pl> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-fs@freebsd.org" Subject: Re: zfs_remove: delete_now case X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Oct 2012 09:34:14 -0000 on 07/10/2012 12:20 Pawel Jakub Dawidek said the following: > I'm fully aware of this code path being dead on FreeBSD. It is left there > only to minimize diff against vendor, so I'd prefer not to remove it. > Surrounding the code with '#ifdef sun' or similar is ok, I think. Ah, very good! It wasn't just clear from the code. I'll try to make the ifdef patch. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Sun Oct 7 15:32:12 2012 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E45B2106566C; Sun, 7 Oct 2012 15:32:12 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id CC9908FC16; Sun, 7 Oct 2012 15:32:11 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA15154; Sun, 07 Oct 2012 18:32:05 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TKspp-0008Hl-Jh; Sun, 07 Oct 2012 18:32:05 +0300 Message-ID: <5071A071.1020800@FreeBSD.org> Date: Sun, 07 Oct 2012 18:32:01 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: "Justin T. Gibbs" , Pawel Jakub Dawidek , Konstantin Belousov References: <76CBA055-021F-458D-8978-E9A973D9B783@scsiguy.com> <506EB43B.8050204@FreeBSD.org> In-Reply-To: <506EB43B.8050204@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: fs@FreeBSD.org Subject: Re: ZFS: Deadlock during vnode recycling X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Oct 2012 15:32:13 -0000 In fact here is a real patch that I would like to propose: http://people.freebsd.org/~avg/zfs-getnewvnode_reserve.diff The patch incorporates the kib's patch for extending VFS API that helps to avoid entering the vnode inactive/reclaim path from getnewvnode. The patch should fix the problem reported in this thread and the problem from "panic: _sx_xlock_hard: recursed on non-recursive sx zfsvfs->z_hold_mtx ..." thread that runs in parallel. Reviews and testing are welcome. Here is a draft of a commit message that also provides some additional details: zfs: overhaul zfs_freebsd_reclaim and zfs_zget... now that we do not need to fear recursion from getnewvnode into zfs_inactive and zfs_freebsd_reclaim. This removes the need for the delayed destruction of znodes via taskqueue, thus making znode/vnode state machine a bit simpler. Also, try to make zfs_zget saner with respected to doomed vnodes to avoid a deadlock when zfs_zget is called from zfs_freebsd_recycle. To do: pass locking flags parameter to zfs_zget, so that the zfs-vfs glue code doesn't have to re-lock a vnode but could ask for proper locking from the very start. The patch also drops some redundant interlock acquisitions, since both vop_inactive and vop_reclaim are called with exclusive vnode lock held. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Sun Oct 7 18:34:56 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0B687106566B; Sun, 7 Oct 2012 18:34:56 +0000 (UTC) (envelope-from gibbs@scsiguy.com) Received: from aslan.scsiguy.com (www.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id A9F388FC14; Sun, 7 Oct 2012 18:34:55 +0000 (UTC) Received: from macbook.scsiguy.com (macbook.scsiguy.com [192.168.0.99]) (authenticated bits=0) by aslan.scsiguy.com (8.14.5/8.14.5) with ESMTP id q97IYnnY084421 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sun, 7 Oct 2012 12:34:49 -0600 (MDT) (envelope-from gibbs@scsiguy.com) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) From: "Justin T. Gibbs" In-Reply-To: <20121003085326.GC1386@garage.freebsd.pl> Date: Sun, 7 Oct 2012 12:34:52 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <505DE715.8020806@FreeBSD.org> <20121003085326.GC1386@garage.freebsd.pl> To: Pawel Jakub Dawidek X-Mailer: Apple Mail (2.1499) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (aslan.scsiguy.com [192.168.0.4]); Sun, 07 Oct 2012 12:34:49 -0600 (MDT) Cc: freebsd-fs@freebsd.org, Andriy Gapon Subject: Re: zfs: allow to mount root from a pool not in zpool.cache X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Oct 2012 18:34:56 -0000 On Oct 3, 2012, at 2:53 AM, Pawel Jakub Dawidek wrote: > On Sat, Sep 22, 2012 at 10:59:56PM -0600, Justin T. Gibbs wrote: >> On Sep 22, 2012, at 10:28 AM, Andriy Gapon wrote: >>=20 >>>=20 >>> Currently FreeBSD ZFS kernel code doesn't allow to mount root = filesystem on a >>> pool that is not listed in zpool.cache as only pools from the cache = are known to >>> ZFS at that time. >>=20 >> I've for some time been of the opinion that FreeBSD should only use >> the cache file for ZFS pools created from non-GEOM objects (i.e. >> files). GEOM tasting should be used to make the kernel aware of >> all pools whether they be imported on the system, partial, or >> foreign. Even for pools created by files, the user land utilities >> should do nothing more than ask the kernel to "taste them". This >> would remove code duplicated in user land for this task (code that >> must be re-executed in kernel space for validation reasons anyway) >> and also help solve problems we've encountered at Spectra with races >> in fault event processing, spare management, and device arrival and >> departures. >>=20 >> So I'm excited by your work in this area and would encourage you >> to "think larger" than just trying to integrate root pool discovery >> with GEOM. Spectra may even be able to help in this work sometime >> in the near future. >=20 > GEOM tasting would most likely require rewriting the code heavly. > Also note that you can have pools in you system that do match your > hostid, but user decided to keep exported and such pool should not be > configured automatically. Not a huge problem probably as there is pool > status somewhere in the metadata that we can use to see if the pool is > exported or not. This topic came up during ZFS day last week. It turns out that the OS-X port of ZFS already does this and Don Brady said he's happy to share patches. I don't see any reason why this type of solution cannot be upstreamed as well. On Illumos, user land can maintain the cache file and use it to ask the in-kernel ZFS code to "taste" devices, minimizing the amount of divergence. -- Justin= From owner-freebsd-fs@FreeBSD.ORG Sun Oct 7 18:43:59 2012 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5A0901065670; Sun, 7 Oct 2012 18:43:59 +0000 (UTC) (envelope-from gibbs@scsiguy.com) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 255A38FC08; Sun, 7 Oct 2012 18:43:58 +0000 (UTC) Received: from macbook.scsiguy.com (macbook.scsiguy.com [192.168.0.99]) (authenticated bits=0) by aslan.scsiguy.com (8.14.5/8.14.5) with ESMTP id q97Ihw7r084465 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sun, 7 Oct 2012 12:43:58 -0600 (MDT) (envelope-from gibbs@scsiguy.com) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) From: "Justin T. Gibbs" In-Reply-To: <5071A071.1020800@FreeBSD.org> Date: Sun, 7 Oct 2012 12:44:02 -0600 Content-Transfer-Encoding: 7bit Message-Id: <97D56E7F-1284-4FB1-8C83-9EE04FE4F59F@scsiguy.com> References: <76CBA055-021F-458D-8978-E9A973D9B783@scsiguy.com> <506EB43B.8050204@FreeBSD.org> <5071A071.1020800@FreeBSD.org> To: Andriy Gapon X-Mailer: Apple Mail (2.1499) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (aslan.scsiguy.com [192.168.0.4]); Sun, 07 Oct 2012 12:43:58 -0600 (MDT) Cc: Konstantin Belousov , Pawel Jakub Dawidek , fs@FreeBSD.org Subject: Re: ZFS: Deadlock during vnode recycling X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Oct 2012 18:43:59 -0000 On Oct 7, 2012, at 9:32 AM, Andriy Gapon wrote: > In fact here is a real patch that I would like to propose: > http://people.freebsd.org/~avg/zfs-getnewvnode_reserve.diff OS-X has these same types of problems and I talked with Don Brady of the OS-X ZFS port about them during ZFS day. It sounds like he explicitly pre-allocates vnodes in these code paths instead of relying on a reserve pool. I plan to review his work since I expect he's found and fixed problems we don't even know we have yet. My only complaint with this patch is that it doesn't include stats counters for these rare conditions so that I can validate that the code is exercised during a test suite. Can you merge in the kstat portion of the change I proposed? -- Justin From owner-freebsd-fs@FreeBSD.ORG Sun Oct 7 20:17:07 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 394DD1065673 for ; Sun, 7 Oct 2012 20:17:07 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id AE6AE8FC0C for ; Sun, 7 Oct 2012 20:17:06 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4EALzhcVCDaFvO/2dsb2JhbABFDoYDuhiCIAEBBSMERwszEQUBEwIEVQaIGKYnkVyOOgGCEoESA45uhn2QLoIyV4FAOw X-IronPort-AV: E=Sophos;i="4.80,548,1344225600"; d="c'?scan'208";a="182460992" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 07 Oct 2012 16:17:05 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 5D4FBB3F36; Sun, 7 Oct 2012 16:17:05 -0400 (EDT) Date: Sun, 7 Oct 2012 16:17:05 -0400 (EDT) From: Rick Macklem To: Piete Brooks Message-ID: <2071960851.1864186.1349641025365.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1864185_1441828033.1349641025360" X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE7 (Win)/6.0.10_GA_2692) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FS List , Ilias Marinos , Brooks Davis , Herbert Poeckl Subject: Kerberized NFS/gssd credential cache issue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Oct 2012 20:17:07 -0000 ------=_Part_1864185_1441828033.1349641025360 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Piete Brooks wrote: > I initially took the priorities to be sorted, but it seems that all > just add > one to the score. Is this as planned, or should "++" become "|= 1 << > N" so > that the most important one aleays wins? My intent was that they all count the same, because I don't know if one is more important than another. (A "more important" one could add +N, if we collectively decide what is "more important".) I hope you don't mind, but I thought if this is going to be discussed, it should be on a mailing list, so I've replaced some of the cc's with freebsd-fs@. (I took out the ones I believe will be reading the list.) Everyone, a discussion has been going on w.r.t. an NFS over Kerberos issue, where the gssd can't find the Kerberos credentials cache file because it assumes it uses a name /tmp/krb5cc_, where is the effective uid. Some setups of sshd use different naming, usually a random suffix appended to the above, to differentiate between login sessions, so the credentials cache can be destroyed upon logout. The Linux gssd does a search of directories, using various heuristics to try and guess which file is the most appropriate one. I've coded a function that does something similar. Since I am not a Kerberos wizzard, I don't know how appropriate the heuristics are. I have attached testcc.c, which is the function plus a simple main() to test it. (Once tested, this function would be used in the gssd to select a credentials cache file.) The current code does the following: - Searches a directory for files that satisfy the following: - has "krb5cc_ as a substring of the file's name - is a regular file - is owned by the uid - has a valid tgt in it For each file that satisfies the above, I generate a "rating", which is an attempt at heuristically guessing the most appropriate file, when there is more than one file matching the above: - add one to the rating for each of - not a cross-realm tgt - the principal without realm is the same name as getpwuid(uid)->pw_name - if the realm for the client principal is the preferred realm (the preferred realm and "krb5cc_" substring are arguments and I was assuming the preferred realm will usually be the default realm) Each of these currently counts one towards the rating. If multiple files matching the above gets the same rating, it uses the one that has the tgt that expires later. So, Kerberos wizzards... Should there be other criteria for selecting the file? Should some of the rating checks count for more than others? (They currently each count as 1, although some could count for more.) Personally, I don't like the idea that a uid has multiple credential cache files, since there is no definitive way to select the "correct one" to authenticate a "uid", but it seems unavoidable. Thanks in advance for any comments, rick ------=_Part_1864185_1441828033.1349641025360-- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 8 03:22:12 2012 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C43A8106566B; Mon, 8 Oct 2012 03:22:12 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9768F8FC19; Mon, 8 Oct 2012 03:22:12 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q983MCXR055406; Mon, 8 Oct 2012 03:22:12 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q983MCAZ055402; Mon, 8 Oct 2012 03:22:12 GMT (envelope-from linimon) Date: Mon, 8 Oct 2012 03:22:12 GMT Message-Id: <201210080322.q983MCAZ055402@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-amd64@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/171626: [tmpfs] tmpfs should be noisier when the requested size cannot be allocated X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2012 03:22:12 -0000 Old Synopsis: tmpfs should be noisier when the requested size cannot be allocated New Synopsis: [tmpfs] tmpfs should be noisier when the requested size cannot be allocated Responsible-Changed-From-To: freebsd-amd64->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Mon Oct 8 03:21:50 UTC 2012 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=171626 From owner-freebsd-fs@FreeBSD.ORG Mon Oct 8 11:07:17 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D7BC3106566C for ; Mon, 8 Oct 2012 11:07:17 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id BFEBF8FC25 for ; Mon, 8 Oct 2012 11:07:17 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q98B7Hkl029316 for ; Mon, 8 Oct 2012 11:07:17 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q98B7HDD029314 for freebsd-fs@FreeBSD.org; Mon, 8 Oct 2012 11:07:17 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 8 Oct 2012 11:07:17 GMT Message-Id: <201210081107.q98B7HDD029314@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2012 11:07:17 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/172348 fs [unionfs] umount -f of filesystem in use with readonly o kern/172334 fs [unionfs] unionfs permits recursive union mounts; caus o kern/172259 fs [zfs] [patch] ZFS fails to receive valid snapshots (pa o kern/171626 fs [tmpfs] tmpfs should be noisier when the requested siz o kern/171415 fs [zfs] zfs recv fails with "cannot receive incremental o kern/170945 fs [gpt] disk layout not portable between direct connect o kern/170914 fs [zfs] [patch] Import patchs related with issues 3090 a o kern/170912 fs [zfs] [patch] unnecessarily setting DS_FLAG_INCONSISTE o bin/170778 fs [zfs] [panic] FreeBSD panics randomly o kern/170680 fs [nfs] Multiple NFS Client bug in the FreeBSD 7.4-RELEA o kern/170497 fs [xfs][panic] kernel will panic whenever I ls a mounted o kern/170238 fs [zfs] [panic] Panic when deleting data o kern/169945 fs [zfs] [panic] Kernel panic while importing zpool (afte o kern/169480 fs [zfs] ZFS stalls on heavy I/O o kern/169398 fs [zfs] Can't remove file with permanent error o kern/169339 fs panic while " : > /etc/123" o kern/169319 fs [zfs] zfs resilver can't complete o kern/168947 fs [nfs] [zfs] .zfs/snapshot directory is messed up when o kern/168942 fs [nfs] [hang] nfsd hangs after being restarted (not -HU o kern/168158 fs [zfs] incorrect parsing of sharenfs options in zfs (fs o kern/167979 fs [ufs] DIOCGDINFO ioctl does not work on 8.2 file syste o kern/167977 fs [smbfs] mount_smbfs results are differ when utf-8 or U o kern/167688 fs [fusefs] Incorrect signal handling with direct_io o kern/167685 fs [zfs] ZFS on USB drive prevents shutdown / reboot o kern/167612 fs [portalfs] The portal file system gets stuck inside po o kern/167272 fs [zfs] ZFS Disks reordering causes ZFS to pick the wron o kern/167260 fs [msdosfs] msdosfs disk was mounted the second time whe o kern/167109 fs [zfs] [panic] zfs diff kernel panic Fatal trap 9: gene o kern/167105 fs [nfs] mount_nfs can not handle source exports wiht mor o kern/167067 fs [zfs] [panic] ZFS panics the server o kern/167066 fs [zfs] ZVOLs not appearing in /dev/zvol o kern/167065 fs [zfs] boot fails when a spare is the boot disk o kern/167048 fs [nfs] [patch] RELEASE-9 crash when using ZFS+NULLFS+NF o kern/166912 fs [ufs] [panic] Panic after converting Softupdates to jo o kern/166851 fs [zfs] [hang] Copying directory from the mounted UFS di o kern/166477 fs [nfs] NFS data corruption. o kern/165950 fs [ffs] SU+J and fsck problem o kern/165923 fs [nfs] Writing to NFS-backed mmapped files fails if flu o kern/165521 fs [zfs] [hang] livelock on 1 Gig of RAM with zfs when 31 o kern/165392 fs Multiple mkdir/rmdir fails with errno 31 o kern/165087 fs [unionfs] lock violation in unionfs o kern/164472 fs [ufs] fsck -B panics on particular data inconsistency o kern/164370 fs [zfs] zfs destroy for snapshot fails on i386 and sparc o kern/164261 fs [nullfs] [patch] fix panic with NFS served from NULLFS o kern/164256 fs [zfs] device entry for volume is not created after zfs o kern/164184 fs [ufs] [panic] Kernel panic with ufs_makeinode o kern/163801 fs [md] [request] allow mfsBSD legacy installed in 'swap' o kern/163770 fs [zfs] [hang] LOR between zfs&syncer + vnlru leading to o kern/163501 fs [nfs] NFS exporting a dir and a subdir in that dir to o kern/162944 fs [coda] Coda file system module looks broken in 9.0 o kern/162860 fs [zfs] Cannot share ZFS filesystem to hosts with a hyph o kern/162751 fs [zfs] [panic] kernel panics during file operations o kern/162591 fs [nullfs] cross-filesystem nullfs does not work as expe o kern/162519 fs [zfs] "zpool import" relies on buggy realpath() behavi o kern/162362 fs [snapshots] [panic] ufs with snapshot(s) panics when g o kern/161968 fs [zfs] [hang] renaming snapshot with -r including a zvo p kern/161897 fs [zfs] [patch] zfs partition probing causing long delay o kern/161864 fs [ufs] removing journaling from UFS partition fails on o bin/161807 fs [patch] add option for explicitly specifying metadata o kern/161579 fs [smbfs] FreeBSD sometimes panics when an smb share is o kern/161533 fs [zfs] [panic] zfs receive panic: system ioctl returnin o kern/161438 fs [zfs] [panic] recursed on non-recursive spa_namespace_ o kern/161424 fs [nullfs] __getcwd() calls fail when used on nullfs mou o kern/161280 fs [zfs] Stack overflow in gptzfsboot o kern/161205 fs [nfs] [pfsync] [regression] [build] Bug report freebsd o kern/161169 fs [zfs] [panic] ZFS causes kernel panic in dbuf_dirty o kern/161112 fs [ufs] [lor] filesystem LOR in FreeBSD 9.0-BETA3 o kern/160893 fs [zfs] [panic] 9.0-BETA2 kernel panic o kern/160860 fs [ufs] Random UFS root filesystem corruption with SU+J o kern/160801 fs [zfs] zfsboot on 8.2-RELEASE fails to boot from root-o o kern/160790 fs [fusefs] [panic] VPUTX: negative ref count with FUSE o kern/160777 fs [zfs] [hang] RAID-Z3 causes fatal hang upon scrub/impo o kern/160706 fs [zfs] zfs bootloader fails when a non-root vdev exists o kern/160591 fs [zfs] Fail to boot on zfs root with degraded raidz2 [r o kern/160410 fs [smbfs] [hang] smbfs hangs when transferring large fil o kern/160283 fs [zfs] [patch] 'zfs list' does abort in make_dataset_ha o kern/159930 fs [ufs] [panic] kernel core o kern/159402 fs [zfs][loader] symlinks cause I/O errors o kern/159357 fs [zfs] ZFS MAXNAMELEN macro has confusing name (off-by- o kern/159356 fs [zfs] [patch] ZFS NAME_ERR_DISKLIKE check is Solaris-s o kern/159351 fs [nfs] [patch] - divide by zero in mountnfs() o kern/159251 fs [zfs] [request]: add FLETCHER4 as DEDUP hash option o kern/159077 fs [zfs] Can't cd .. with latest zfs version o kern/159048 fs [smbfs] smb mount corrupts large files o kern/159045 fs [zfs] [hang] ZFS scrub freezes system o kern/158839 fs [zfs] ZFS Bootloader Fails if there is a Dead Disk o kern/158802 fs amd(8) ICMP storm and unkillable process. o kern/158231 fs [nullfs] panic on unmounting nullfs mounted over ufs o f kern/157929 fs [nfs] NFS slow read o kern/157399 fs [zfs] trouble with: mdconfig force delete && zfs strip o kern/157179 fs [zfs] zfs/dbuf.c: panic: solaris assert: arc_buf_remov o kern/156797 fs [zfs] [panic] Double panic with FreeBSD 9-CURRENT and o kern/156781 fs [zfs] zfs is losing the snapshot directory, p kern/156545 fs [ufs] mv could break UFS on SMP systems o kern/156193 fs [ufs] [hang] UFS snapshot hangs && deadlocks processes o kern/156039 fs [nullfs] [unionfs] nullfs + unionfs do not compose, re o kern/155615 fs [zfs] zfs v28 broken on sparc64 -current o kern/155587 fs [zfs] [panic] kernel panic with zfs p kern/155411 fs [regression] [8.2-release] [tmpfs]: mount: tmpfs : No o kern/155199 fs [ext2fs] ext3fs mounted as ext2fs gives I/O errors o bin/155104 fs [zfs][patch] use /dev prefix by default when importing o kern/154930 fs [zfs] cannot delete/unlink file from full volume -> EN o kern/154828 fs [msdosfs] Unable to create directories on external USB o kern/154491 fs [smbfs] smb_co_lock: recursive lock for object 1 p kern/154228 fs [md] md getting stuck in wdrain state o kern/153996 fs [zfs] zfs root mount error while kernel is not located o kern/153753 fs [zfs] ZFS v15 - grammatical error when attempting to u o kern/153716 fs [zfs] zpool scrub time remaining is incorrect o kern/153695 fs [patch] [zfs] Booting from zpool created on 4k-sector o kern/153680 fs [xfs] 8.1 failing to mount XFS partitions o kern/153520 fs [zfs] Boot from GPT ZFS root on HP BL460c G1 unstable o kern/153418 fs [zfs] [panic] Kernel Panic occurred writing to zfs vol o kern/153351 fs [zfs] locking directories/files in ZFS o bin/153258 fs [patch][zfs] creating ZVOLs requires `refreservation' s kern/153173 fs [zfs] booting from a gzip-compressed dataset doesn't w o bin/153142 fs [zfs] ls -l outputs `ls: ./.zfs: Operation not support o kern/153126 fs [zfs] vdev failure, zpool=peegel type=vdev.too_small o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/151111 fs [zfs] vnodes leakage during zfs unmount o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " p kern/147560 fs [zfs] [boot] Booting 8.1-PRERELEASE raidz system take o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an f bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs p bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic p kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o kern/118318 fs [nfs] NFS server hangs under special circumstances o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118126 fs [nfs] [patch] Poor NFS server write performance o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116583 fs [ffs] [hang] System freezes for short time when using o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 293 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 8 13:11:50 2012 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5AB6610656D1; Mon, 8 Oct 2012 13:11:50 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay04.ispgateway.de (smtprelay04.ispgateway.de [80.67.31.27]) by mx1.freebsd.org (Postfix) with ESMTP id 14F908FC16; Mon, 8 Oct 2012 13:11:49 +0000 (UTC) Received: from [84.44.210.110] (helo=fabiankeil.de) by smtprelay04.ispgateway.de with esmtpsa (SSLv3:AES128-SHA:128) (Exim 4.68) (envelope-from ) id 1TLCym-0006VS-F3; Mon, 08 Oct 2012 15:02:40 +0200 Date: Mon, 8 Oct 2012 14:35:41 +0200 From: Fabian Keil To: Andriy Gapon Message-ID: <20121008143541.537d8490@fabiankeil.de> In-Reply-To: <5071A071.1020800@FreeBSD.org> References: <76CBA055-021F-458D-8978-E9A973D9B783@scsiguy.com> <506EB43B.8050204@FreeBSD.org> <5071A071.1020800@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/T+pdA.9Hn4G.E1AymhzME8V"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 Cc: fs@FreeBSD.org Subject: Re: ZFS: Deadlock during vnode recycling X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2012 13:11:50 -0000 --Sig_/T+pdA.9Hn4G.E1AymhzME8V Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Andriy Gapon wrote: > In fact here is a real patch that I would like to propose: > http://people.freebsd.org/~avg/zfs-getnewvnode_reserve.diff >=20 > The patch incorporates the kib's patch for extending VFS API that helps t= o avoid > entering the vnode inactive/reclaim path from getnewvnode. >=20 > The patch should fix the problem reported in this thread and the problem = from > "panic: _sx_xlock_hard: recursed on non-recursive sx zfsvfs->z_hold_mtx .= .." > thread that runs in parallel. >=20 > Reviews and testing are welcome. I'd probably replace the intentionally empty "else if (zp->z_unlinked)" block in zfs_freebsd_reclaim() with a comment, but other than that the patch looks good to me. While I'm using nullfs on a couple of jails, I never experienced the panic above and can thus only report that I didn't notice any regressions so far since applying the patch yesterday. Fabian --Sig_/T+pdA.9Hn4G.E1AymhzME8V Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlByyKAACgkQBYqIVf93VJ01ZQCgtVvK7AyF2rgoEvAk0+wpALS/ f/8An1RtXtLDyv83IFRZZ0YX8pCcTq0k =7PFl -----END PGP SIGNATURE----- --Sig_/T+pdA.9Hn4G.E1AymhzME8V-- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 8 14:57:41 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 247EB106564A; Mon, 8 Oct 2012 14:57:41 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 551168FC12; Mon, 8 Oct 2012 14:57:40 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id e12so2771403lag.13 for ; Mon, 08 Oct 2012 07:57:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=28K7FgCY6rXSwtJ5BxO7jwycuhZF7shiABOISj2/WUY=; b=LSBKdQaD/PFle7ZbYfnV3lBn3cDIjWduawWkpzEMXNsZ5m/0euJIchCZGReg+Gxdl+ Baa9bJIxRVL+TesV6DXeaaiwvP0jmLnlNdd0ev33eqWrkjNH1CiVRdUoyo3OzsCo6UgB 9dfhYq+q1Qop208FMKqkF9ifQrirgp+j7agdyXpLXa6rK44Cy7qHvVEOBIfYU1sMx9p4 SWO9CYF+FmcflHUmAn5JH28qEifEGc0VyZy/V434kALqV1IR0VboELrid55jlaMZQRzc 6X0BmEXUr4n69xiQohsUtAoWxujtmsz0bJbqlUQfha6U+99am6G6FamO/I3+t2ZpX9jl Ek2w== MIME-Version: 1.0 Received: by 10.112.26.135 with SMTP id l7mr6904608lbg.84.1349708259245; Mon, 08 Oct 2012 07:57:39 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.112.101.234 with HTTP; Mon, 8 Oct 2012 07:57:39 -0700 (PDT) In-Reply-To: <5065B873.4070509@omnilan.de> References: <20120829060158.GA38721@x2.osted.lan> <20120831052003.GA91340@x2.osted.lan> <20120905201531.GA54452@x2.osted.lan> <20120917140055.GA9037@x2.osted.lan> <5061F6E9.6030104@omnilan.de> <5062E0DE.70805@omnilan.de> <5065B873.4070509@omnilan.de> Date: Mon, 8 Oct 2012 16:57:39 +0200 X-Google-Sender-Auth: sbekD6Y3GDejF1fO13_vgqMipis Message-ID: From: Attilio Rao To: Harald Schmalzbauer Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD FS , freebsd-current@freebsd.org Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: attilio@FreeBSD.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2012 14:57:41 -0000 On Fri, Sep 28, 2012 at 4:47 PM, Harald Schmalzbauer wrote: > schrieb Attilio Rao am 28.09.2012 16:18 (localtime): >> On Wed, Sep 26, 2012 at 12:02 PM, Harald Schmalzbauer >> wrote: >>> ... >> After many people willing to test fuse on STABLE_9, I made this patch >> that at least compiles there: >> http://www.freebsd.org/~attilio/fuse_import/fuse_stable9_241030.patch > > Thanks a lot! In the meantime I made the original patch compiling. I > simply looked at the changes which were made around july in the fuse > project to follow changes in head (checkpath(), vrecycle() and > vtruncbuf()) and "reverted" them. > Since I have no idea about the code I modified, I'm happy that you did a > more qualified patch set :-) > >> Of course, I didn't have a chance to test it because I'm also out for >> vacation right now but please do and report. > > Happy holiday!!! If you're by chance arround the Oktoberfest, drop me a > note, I'll pay you a Ma=C3=9F (or any other drink if you don't like > =E2=80=9EWiesnbier=E2=80=9C) :-) I really hoped to make this year, but no luck :/ >>> ... >>> Some questions: Is this planned to be mfc'd and if so, how can one know= ? >> In which sense "how can one know?". We usually specify MFC timeouts in >> the commit message (not sure if this answers your concerns). > > Yep, that's what I wanted to know. So if there's no MFC timeout in the > log, it's not intended to be MFCd ever I guess. > > Thanks a lot! > World/Kernel compiled fine in the meantime, I'll do some sshfs tests. Did you do any test in the end? Thanks, Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-fs@FreeBSD.ORG Mon Oct 8 16:39:52 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1D7901065670 for ; Mon, 8 Oct 2012 16:39:52 +0000 (UTC) (envelope-from dustinwenz@ebureau.com) Received: from internet02.ebureau.com (internet02.tru-signal.biz [65.127.24.21]) by mx1.freebsd.org (Postfix) with ESMTP id D56588FC0A for ; Mon, 8 Oct 2012 16:39:51 +0000 (UTC) Received: from service02.office.ebureau.com (internet06.ebureau.com [65.127.24.25]) by internet02.ebureau.com (Postfix) with ESMTP id 567F6DDD4BD for ; Mon, 8 Oct 2012 11:32:15 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by service02.office.ebureau.com (Postfix) with ESMTP id 521368D3EB6 for ; Mon, 8 Oct 2012 11:32:15 -0500 (CDT) X-Virus-Scanned: amavisd-new at ebureau.com Received: from service02.office.ebureau.com ([127.0.0.1]) by localhost (internet06.ebureau.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RsU8UBf8w_Yr for ; Mon, 8 Oct 2012 11:32:15 -0500 (CDT) Received: from square.office.ebureau.com (square.office.ebureau.com [10.10.20.22]) by service02.office.ebureau.com (Postfix) with ESMTPSA id 0ABEF8D3EA9 for ; Mon, 8 Oct 2012 11:32:15 -0500 (CDT) From: Dustin Wenz Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <6116A56E-4565-4485-887E-46E3ED231606@ebureau.com> Date: Mon, 8 Oct 2012 11:32:14 -0500 To: freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) X-Mailer: Apple Mail (2.1498) Subject: Imposing ZFS latency limits X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2012 16:39:52 -0000 I'm trying to find a way to get lower read latency in ZFS when there is = a failing disk in a mirror. The specific failure mode that I'm trying to = handle is when a disk takes significantly longer than usual to respond = to read requests, but is not so far gone that the OS drops it from the = bus. In that case, the whole pool can become much less responsive than I = think it needs to be. Since ZFS has access to the data I'm reading through other members in a = mirror, I would like ZFS to defer to the redundant disks when individual = disk latency exceeds some threshold that I would define. Is there any = provision for this in the FreeBSD ZFS implementation? - .Dustin From owner-freebsd-fs@FreeBSD.ORG Mon Oct 8 17:28:35 2012 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 80C0F1065673 for ; Mon, 8 Oct 2012 17:28:35 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com [98.139.253.104]) by mx1.freebsd.org (Postfix) with ESMTP id 3A1A58FC16 for ; Mon, 8 Oct 2012 17:28:34 +0000 (UTC) Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q98HHxYY078323 for ; Mon, 8 Oct 2012 10:18:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1349716680; bh=5hU5VQTSuyPHD6wDIzZvTGjQjON+aCZz9r9l7kr/kjw=; h=Subject:From:Reply-To:To:Content-Type:Date:Message-ID: Mime-Version:Content-Transfer-Encoding; b=CVJZQnNXxjhdFjlWykWOLYj9NXACAa3hTk6P0DaDY5Lgn9trz+ntk6MoalLAcaDI0 zaH8l9eANaEI6XrgoEMrAapTK3Wb6r9KWzBvAhHRu9FtJSXaM6+LC+k/6kUg5i283C h29b7pmc1anyPg+w1WsI+yKVUT6gP68YqLjSYJYc= From: Sean Bruno To: fs@freebsd.org Content-Type: text/plain; charset="UTF-8" Date: Mon, 08 Oct 2012 10:17:59 -0700 Message-ID: <1349716679.4062.8.camel@powernoodle.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 716680001 Cc: Subject: zfs zlib and net/zlib conflict X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2012 17:28:35 -0000 Hey, noted that DDB_CTF and ZFS are one cause of zlib conflict currently. I guess, for now, I can't have CTF *and* ZFS in my kernconf, which is ok, I guess. I can see that there will be some issues until someone can untangle zlib as sys/conf/files will drag in zlib from net in the following options: net/zlib.c optional crypto | geom_uzip | ipsec | \ mxge | netgraph_deflate | \ ddb_ctf | gzio |geom_uncompress Thoughts? Sean From owner-freebsd-fs@FreeBSD.ORG Mon Oct 8 19:17:35 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C99861065670 for ; Mon, 8 Oct 2012 19:17:35 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com [98.139.253.104]) by mx1.freebsd.org (Postfix) with ESMTP id 8F1268FC0A for ; Mon, 8 Oct 2012 19:17:35 +0000 (UTC) Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q98JHAcr027609 for ; Mon, 8 Oct 2012 12:17:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1349723830; bh=mWtyKDveKVFDHGF/EC/1+dTg+I/MOChmhuKCUtmXfaI=; h=Subject:From:Reply-To:To:Content-Type:Date:Message-ID: Mime-Version:Content-Transfer-Encoding; b=GNkDbTo4q8lFhFI4xR+2Jl0Cufq5ln87uei5kl1LvfmtLtB55xuwunRa54HsmKxob UKhMWBefapibefbh+MX587QNBxyxuVBcTJ09mgXr6TJYCgl95B6zZQdoZkSUJKdh7/ +9ww/rNHCF3/do3jPmsSRw4pBIx8aBSiJyURmmNM= From: Sean Bruno To: freebsd-fs@freebsd.org Content-Type: text/plain; charset="UTF-8" Date: Mon, 08 Oct 2012 12:17:09 -0700 Message-ID: <1349723829.90401.2.camel@powernoodle.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 723830001 Subject: zfs zlib and net/zlib conflict X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2012 19:17:35 -0000 Hey, noted that DDB_CTF and ZFS are one cause of zlib conflict currently. Not sure how to get around it. I guess, for now, I can't have CTF/Dtrace *and* ZFS in my kernconf? I can see that there will be some issues until someone can untangle zlib as sys/conf/files will drag in zlib from net in the following options: net/zlib.c optional crypto | geom_uzip | ipsec | \ mxge | netgraph_deflate | \ ddb_ctf | gzio |geom_uncompress Sean From owner-freebsd-fs@FreeBSD.ORG Mon Oct 8 19:18:39 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A92FF106564A for ; Mon, 8 Oct 2012 19:18:39 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com [98.139.253.104]) by mx1.freebsd.org (Postfix) with ESMTP id 586B68FC12 for ; Mon, 8 Oct 2012 19:18:39 +0000 (UTC) Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q98JINMV028019 for ; Mon, 8 Oct 2012 12:18:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1349723904; bh=vDOicdmn6vN94KTX1y3T8nBm20CYtipC22/ltRNh6fI=; h=Subject:From:Reply-To:To:Content-Type:Date:Message-ID: Mime-Version:Content-Transfer-Encoding; b=ZPFBL3VshPhW3Rpp2nKjkHBkYvCUOBemydHZsmbWb6CC0KqH4aJ8xDKz5dgDEqcah OVR+XXEmnyQDdHm1fGERcqu2qPy6F171SSC2JJR1Um+bcTgjQxQPXjKRb5wnTB9QgQ aWyHWz4bTukhw2FZYWV0PjsXdQSBVOys2gpJ3b2A= From: Sean Bruno To: freebsd-fs Content-Type: text/plain; charset="UTF-8" Date: Mon, 08 Oct 2012 12:18:23 -0700 Message-ID: <1349723903.90401.4.camel@powernoodle.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 723904000 Subject: -current ZFS build on stable/9 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2012 19:18:39 -0000 Not sure how to get around this buildfail at the moment. I'm using generic with options ZFS and I have commented out CTF related entries. ===> zlib (depend) -------------------------------------------------------------- >>> stage 3.2: building everything -------------------------------------------------------------- linking kernel.debug spa.o: In function `spa_deactivate': /home/seanbru/svn_head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1042: undefined reference to `trim_thread_destroy' spa.o: In function `spa_unload': /home/seanbru/svn_head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1162: undefined reference to `trim_thread_destroy' spa.o: In function `spa_activate': /home/seanbru/svn_head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1008: undefined reference to `trim_thread_create' vdev.o: In function `vdev_close': /home/seanbru/svn_head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1449: undefined reference to `trim_map_destroy' vdev.o: In function `vdev_open': /home/seanbru/svn_head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1201: undefined reference to `trim_map_create' vdev_label.o: In function `vdev_config_sync': /home/seanbru/svn_head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c:1299: undefined reference to `trim_thread_wakeup' zio.o: In function `zio_vdev_io_done': /home/seanbru/svn_head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2474: undefined reference to `trim_map_write_done' zio.o: In function `zio_vdev_io_start': /home/seanbru/svn_head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2368: undefined reference to `trim_map_free' /home/seanbru/svn_head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2452: undefined reference to `trim_map_write_start' *** [kernel.debug] Error code 1 Stop in /builder/build/next/home/seanbru/svn_head/sys/GENERICZFS. *** [buildkernel] Error code 1 Stop in /home/seanbru/svn_head. *** [buildkernel] Error code 1 Stop in /home/seanbru/svn_head. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 8 11:07:17 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D7BC3106566C for ; Mon, 8 Oct 2012 11:07:17 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id BFEBF8FC25 for ; Mon, 8 Oct 2012 11:07:17 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q98B7Hkl029316 for ; Mon, 8 Oct 2012 11:07:17 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q98B7HDD029314 for freebsd-fs@FreeBSD.org; Mon, 8 Oct 2012 11:07:17 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 8 Oct 2012 11:07:17 GMT Message-Id: <201210081107.q98B7HDD029314@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2012 11:07:17 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/172348 fs [unionfs] umount -f of filesystem in use with readonly o kern/172334 fs [unionfs] unionfs permits recursive union mounts; caus o kern/172259 fs [zfs] [patch] ZFS fails to receive valid snapshots (pa o kern/171626 fs [tmpfs] tmpfs should be noisier when the requested siz o kern/171415 fs [zfs] zfs recv fails with "cannot receive incremental o kern/170945 fs [gpt] disk layout not portable between direct connect o kern/170914 fs [zfs] [patch] Import patchs related with issues 3090 a o kern/170912 fs [zfs] [patch] unnecessarily setting DS_FLAG_INCONSISTE o bin/170778 fs [zfs] [panic] FreeBSD panics randomly o kern/170680 fs [nfs] Multiple NFS Client bug in the FreeBSD 7.4-RELEA o kern/170497 fs [xfs][panic] kernel will panic whenever I ls a mounted o kern/170238 fs [zfs] [panic] Panic when deleting data o kern/169945 fs [zfs] [panic] Kernel panic while importing zpool (afte o kern/169480 fs [zfs] ZFS stalls on heavy I/O o kern/169398 fs [zfs] Can't remove file with permanent error o kern/169339 fs panic while " : > /etc/123" o kern/169319 fs [zfs] zfs resilver can't complete o kern/168947 fs [nfs] [zfs] .zfs/snapshot directory is messed up when o kern/168942 fs [nfs] [hang] nfsd hangs after being restarted (not -HU o kern/168158 fs [zfs] incorrect parsing of sharenfs options in zfs (fs o kern/167979 fs [ufs] DIOCGDINFO ioctl does not work on 8.2 file syste o kern/167977 fs [smbfs] mount_smbfs results are differ when utf-8 or U o kern/167688 fs [fusefs] Incorrect signal handling with direct_io o kern/167685 fs [zfs] ZFS on USB drive prevents shutdown / reboot o kern/167612 fs [portalfs] The portal file system gets stuck inside po o kern/167272 fs [zfs] ZFS Disks reordering causes ZFS to pick the wron o kern/167260 fs [msdosfs] msdosfs disk was mounted the second time whe o kern/167109 fs [zfs] [panic] zfs diff kernel panic Fatal trap 9: gene o kern/167105 fs [nfs] mount_nfs can not handle source exports wiht mor o kern/167067 fs [zfs] [panic] ZFS panics the server o kern/167066 fs [zfs] ZVOLs not appearing in /dev/zvol o kern/167065 fs [zfs] boot fails when a spare is the boot disk o kern/167048 fs [nfs] [patch] RELEASE-9 crash when using ZFS+NULLFS+NF o kern/166912 fs [ufs] [panic] Panic after converting Softupdates to jo o kern/166851 fs [zfs] [hang] Copying directory from the mounted UFS di o kern/166477 fs [nfs] NFS data corruption. o kern/165950 fs [ffs] SU+J and fsck problem o kern/165923 fs [nfs] Writing to NFS-backed mmapped files fails if flu o kern/165521 fs [zfs] [hang] livelock on 1 Gig of RAM with zfs when 31 o kern/165392 fs Multiple mkdir/rmdir fails with errno 31 o kern/165087 fs [unionfs] lock violation in unionfs o kern/164472 fs [ufs] fsck -B panics on particular data inconsistency o kern/164370 fs [zfs] zfs destroy for snapshot fails on i386 and sparc o kern/164261 fs [nullfs] [patch] fix panic with NFS served from NULLFS o kern/164256 fs [zfs] device entry for volume is not created after zfs o kern/164184 fs [ufs] [panic] Kernel panic with ufs_makeinode o kern/163801 fs [md] [request] allow mfsBSD legacy installed in 'swap' o kern/163770 fs [zfs] [hang] LOR between zfs&syncer + vnlru leading to o kern/163501 fs [nfs] NFS exporting a dir and a subdir in that dir to o kern/162944 fs [coda] Coda file system module looks broken in 9.0 o kern/162860 fs [zfs] Cannot share ZFS filesystem to hosts with a hyph o kern/162751 fs [zfs] [panic] kernel panics during file operations o kern/162591 fs [nullfs] cross-filesystem nullfs does not work as expe o kern/162519 fs [zfs] "zpool import" relies on buggy realpath() behavi o kern/162362 fs [snapshots] [panic] ufs with snapshot(s) panics when g o kern/161968 fs [zfs] [hang] renaming snapshot with -r including a zvo p kern/161897 fs [zfs] [patch] zfs partition probing causing long delay o kern/161864 fs [ufs] removing journaling from UFS partition fails on o bin/161807 fs [patch] add option for explicitly specifying metadata o kern/161579 fs [smbfs] FreeBSD sometimes panics when an smb share is o kern/161533 fs [zfs] [panic] zfs receive panic: system ioctl returnin o kern/161438 fs [zfs] [panic] recursed on non-recursive spa_namespace_ o kern/161424 fs [nullfs] __getcwd() calls fail when used on nullfs mou o kern/161280 fs [zfs] Stack overflow in gptzfsboot o kern/161205 fs [nfs] [pfsync] [regression] [build] Bug report freebsd o kern/161169 fs [zfs] [panic] ZFS causes kernel panic in dbuf_dirty o kern/161112 fs [ufs] [lor] filesystem LOR in FreeBSD 9.0-BETA3 o kern/160893 fs [zfs] [panic] 9.0-BETA2 kernel panic o kern/160860 fs [ufs] Random UFS root filesystem corruption with SU+J o kern/160801 fs [zfs] zfsboot on 8.2-RELEASE fails to boot from root-o o kern/160790 fs [fusefs] [panic] VPUTX: negative ref count with FUSE o kern/160777 fs [zfs] [hang] RAID-Z3 causes fatal hang upon scrub/impo o kern/160706 fs [zfs] zfs bootloader fails when a non-root vdev exists o kern/160591 fs [zfs] Fail to boot on zfs root with degraded raidz2 [r o kern/160410 fs [smbfs] [hang] smbfs hangs when transferring large fil o kern/160283 fs [zfs] [patch] 'zfs list' does abort in make_dataset_ha o kern/159930 fs [ufs] [panic] kernel core o kern/159402 fs [zfs][loader] symlinks cause I/O errors o kern/159357 fs [zfs] ZFS MAXNAMELEN macro has confusing name (off-by- o kern/159356 fs [zfs] [patch] ZFS NAME_ERR_DISKLIKE check is Solaris-s o kern/159351 fs [nfs] [patch] - divide by zero in mountnfs() o kern/159251 fs [zfs] [request]: add FLETCHER4 as DEDUP hash option o kern/159077 fs [zfs] Can't cd .. with latest zfs version o kern/159048 fs [smbfs] smb mount corrupts large files o kern/159045 fs [zfs] [hang] ZFS scrub freezes system o kern/158839 fs [zfs] ZFS Bootloader Fails if there is a Dead Disk o kern/158802 fs amd(8) ICMP storm and unkillable process. o kern/158231 fs [nullfs] panic on unmounting nullfs mounted over ufs o f kern/157929 fs [nfs] NFS slow read o kern/157399 fs [zfs] trouble with: mdconfig force delete && zfs strip o kern/157179 fs [zfs] zfs/dbuf.c: panic: solaris assert: arc_buf_remov o kern/156797 fs [zfs] [panic] Double panic with FreeBSD 9-CURRENT and o kern/156781 fs [zfs] zfs is losing the snapshot directory, p kern/156545 fs [ufs] mv could break UFS on SMP systems o kern/156193 fs [ufs] [hang] UFS snapshot hangs && deadlocks processes o kern/156039 fs [nullfs] [unionfs] nullfs + unionfs do not compose, re o kern/155615 fs [zfs] zfs v28 broken on sparc64 -current o kern/155587 fs [zfs] [panic] kernel panic with zfs p kern/155411 fs [regression] [8.2-release] [tmpfs]: mount: tmpfs : No o kern/155199 fs [ext2fs] ext3fs mounted as ext2fs gives I/O errors o bin/155104 fs [zfs][patch] use /dev prefix by default when importing o kern/154930 fs [zfs] cannot delete/unlink file from full volume -> EN o kern/154828 fs [msdosfs] Unable to create directories on external USB o kern/154491 fs [smbfs] smb_co_lock: recursive lock for object 1 p kern/154228 fs [md] md getting stuck in wdrain state o kern/153996 fs [zfs] zfs root mount error while kernel is not located o kern/153753 fs [zfs] ZFS v15 - grammatical error when attempting to u o kern/153716 fs [zfs] zpool scrub time remaining is incorrect o kern/153695 fs [patch] [zfs] Booting from zpool created on 4k-sector o kern/153680 fs [xfs] 8.1 failing to mount XFS partitions o kern/153520 fs [zfs] Boot from GPT ZFS root on HP BL460c G1 unstable o kern/153418 fs [zfs] [panic] Kernel Panic occurred writing to zfs vol o kern/153351 fs [zfs] locking directories/files in ZFS o bin/153258 fs [patch][zfs] creating ZVOLs requires `refreservation' s kern/153173 fs [zfs] booting from a gzip-compressed dataset doesn't w o bin/153142 fs [zfs] ls -l outputs `ls: ./.zfs: Operation not support o kern/153126 fs [zfs] vdev failure, zpool=peegel type=vdev.too_small o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/151111 fs [zfs] vnodes leakage during zfs unmount o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " p kern/147560 fs [zfs] [boot] Booting 8.1-PRERELEASE raidz system take o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an f bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs p bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic p kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o kern/118318 fs [nfs] NFS server hangs under special circumstances o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118126 fs [nfs] [patch] Poor NFS server write performance o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116583 fs [ffs] [hang] System freezes for short time when using o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 293 problems total. From owner-freebsd-fs@FreeBSD.ORG Tue Oct 9 09:11:25 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9516A631 for ; Tue, 9 Oct 2012 09:11:25 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 795EA8FC0C for ; Tue, 9 Oct 2012 09:11:23 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA05249 for ; Tue, 09 Oct 2012 12:11:22 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TLVqU-000GLF-I4 for freebsd-fs@freebsd.org; Tue, 09 Oct 2012 12:11:22 +0300 Resent-From: Andriy Gapon Resent-To: "freebsd-fs@freebsd.org" Resent-Date: Tue, 9 Oct 2012 12:11:21 +0300 Resent-Message-Id: <5073EA39.7080403@FreeBSD.org> Resent-User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 Message-ID: <5073E086.8090508@FreeBSD.org> Date: Tue, 09 Oct 2012 11:29:58 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: "Justin T. Gibbs" , Pawel Jakub Dawidek , Konstantin Belousov Subject: Re: ZFS: Deadlock during vnode recycling References: <76CBA055-021F-458D-8978-E9A973D9B783@scsiguy.com> <506EB43B.8050204@FreeBSD.org> <5071A071.1020800@FreeBSD.org> In-Reply-To: <5071A071.1020800@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2012 09:11:25 -0000 on 07/10/2012 18:32 Andriy Gapon said the following: > In fact here is a real patch that I would like to propose: > http://people.freebsd.org/~avg/zfs-getnewvnode_reserve.diff > > The patch incorporates the kib's patch for extending VFS API that helps to avoid > entering the vnode inactive/reclaim path from getnewvnode. > > The patch should fix the problem reported in this thread and the problem from > "panic: _sx_xlock_hard: recursed on non-recursive sx zfsvfs->z_hold_mtx ..." > thread that runs in parallel. > > Reviews and testing are welcome. > > Here is a draft of a commit message that also provides some additional details: > > zfs: overhaul zfs_freebsd_reclaim and zfs_zget... > > now that we do not need to fear recursion from getnewvnode into > zfs_inactive and zfs_freebsd_reclaim. > This removes the need for the delayed destruction of znodes via taskqueue, > thus making znode/vnode state machine a bit simpler. > Also, try to make zfs_zget saner with respected to doomed vnodes to avoid > a deadlock when zfs_zget is called from zfs_freebsd_recycle. > > To do: pass locking flags parameter to zfs_zget, so that the zfs-vfs glue > code doesn't have to re-lock a vnode but could ask for proper locking > from the very start. > > > The patch also drops some redundant interlock acquisitions, since both > vop_inactive and vop_reclaim are called with exclusive vnode lock held. Turns out that it is a bad idea to get vnode lock in zfs_zget (e.g. via vn_lock or vget). If we come to zfs_zget from zil_commit it may happen that we may try to grab a lock of a vnode that is already locked by a different thread and that thread might be waiting for us to exit zil_commit. Hence a deadlock between the threads. Here is an updated patch: http://people.freebsd.org/~avg/zfs-getnewvnode_reserve.1.diff It reverts part of zfs_zget logic closer to the original code (no vget call). -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Oct 9 09:11:41 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5051E633 for ; Tue, 9 Oct 2012 09:11:41 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 85CC38FC14 for ; Tue, 9 Oct 2012 09:11:40 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA05257 for ; Tue, 09 Oct 2012 12:11:39 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TLVqk-000GLI-PA for freebsd-fs@freebsd.org; Tue, 09 Oct 2012 12:11:38 +0300 Resent-From: Andriy Gapon Resent-To: "freebsd-fs@freebsd.org" Resent-Date: Tue, 9 Oct 2012 12:11:38 +0300 Resent-Message-Id: <5073EA4A.5020100@FreeBSD.org> Resent-User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 Message-ID: <5073E136.5090707@FreeBSD.org> Date: Tue, 09 Oct 2012 11:32:54 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: "Justin T. Gibbs" Subject: Re: ZFS: Deadlock during vnode recycling References: <76CBA055-021F-458D-8978-E9A973D9B783@scsiguy.com> <506EB43B.8050204@FreeBSD.org> <5071A071.1020800@FreeBSD.org> <97D56E7F-1284-4FB1-8C83-9EE04FE4F59F@scsiguy.com> In-Reply-To: <97D56E7F-1284-4FB1-8C83-9EE04FE4F59F@scsiguy.com> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2012 09:11:41 -0000 on 07/10/2012 21:44 Justin T. Gibbs said the following: > On Oct 7, 2012, at 9:32 AM, Andriy Gapon wrote: > >> In fact here is a real patch that I would like to propose: >> http://people.freebsd.org/~avg/zfs-getnewvnode_reserve.diff > > OS-X has these same types of problems and I talked with Don Brady > of the OS-X ZFS port about them during ZFS day. It sounds like he > explicitly pre-allocates vnodes in these code paths instead of > relying on a reserve pool. How is pre-allocation different from reservation? Terminologically it looks like different words for the same thing. > I plan to review his work since I expect > he's found and fixed problems we don't even know we have yet. That would be very interesting to learn. > My only complaint with this patch is that it doesn't include stats > counters for these rare conditions so that I can validate that the > code is exercised during a test suite. Can you merge in the kstat > portion of the change I proposed? Umm, I was not planning on doing that... Your last patch looked rather big. If we decide that including those stats in that way into the tree is a way to go, then I will try to look into doing that. Unless you beat me to it. P.S. it is not fact that the patch of mine could go into the tree, of course :-) -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Oct 9 12:47:36 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A5D8D785 for ; Tue, 9 Oct 2012 12:47:36 +0000 (UTC) (envelope-from root@najem.faculty.kuniv.edu) Received: from najem.faculty.kuniv.edu (najem.faculty.kuniv.edu [139.141.148.115]) by mx1.freebsd.org (Postfix) with SMTP id E4A748FC12 for ; Tue, 9 Oct 2012 12:47:35 +0000 (UTC) Received: by najem.faculty.kuniv.edu (Postfix, from userid 0) id 6F0E37F0A1; Tue, 9 Oct 2012 15:34:50 +0300 (AST) To: freebsd-fs@freebsd.org Subject: Concours - lots d'une valeur commerciale totale de 8 920 E From: " Banque Postale " Content-Type: text/plain Message-Id: <20121009123450.6F0E37F0A1@najem.faculty.kuniv.edu> Date: Tue, 9 Oct 2012 15:34:50 +0300 (AST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2012 12:47:36 -0000 Éligibilité : Ce Jeu s'adresse à toute personne physique résidant en France métropolitaine à l'exclusion des personnels de la Société Organisatrice et de l'Opérateur, ainsi qu'aux membres de leur famille. Durée du concours : Le Concours se déroule jusqu'au 10 octobre 2012 inclus. Limite : Un seul et unique compte de joueur sera ouvert par foyer possédant les mêmes noms, adresse électronique et domicile pendant toute la durée du Jeu. Tirages : Le tirage au sort aura lieu le 10 octobre 2012. Liste des dotations : Ce Jeu est doté de 10 lots d.une valeur commerciale totale de 8 920 . TTC répartie de la façon suivante : 6 CA DO Chèques La Banque Postale d'une valeur unitaire maximum de 100. TTC 1 casque Hi-fi stéréo de la marque Bose d'une valeur unitaire maximum de 200. 2 paires d.enceintes Micropod de la marque Scandyna d.un prix unitaire maximum de 200. TTC 2 caméscopes numériques Haute Définition de poche de la marque Sony d'une valeur unitaire maximum de 250. TTC 2 Home cinéma Blu Ray de la marque LG d'une valeur unitaire maximum de 280. 2 Ecrans plats Haute Définition 23 pouces de la marque Samsung d'une valeur unitaire maximum de 300. TTC 1 iPad 16 Go wifi de la marque Apple d'une valeur unitaire maximum de 500. TTC 2 iPhone 4G 16 Go de la marque Apple d'une valeur unitaire maximum de 630. TTC 2 appareils photo Reflex D3000 de la marque Nikon d'une valeur unitaire maximum de 650. TTC 1 ordinateur portable Macbook 2.4Ghz de la marque Apple d'une valeur unitaire maximum de 1000. TTC Inscrivez-vous pour participer au tirage au sort! Pour participer, cliquez sur le lien ci-dessous pour accéder à votre page de connexion: http://www.lbpstlefr.net/wsost/?concours Bonne Chance, Votre Service Client La Banque Postale From owner-freebsd-fs@FreeBSD.ORG Tue Oct 9 21:28:56 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BD1C427D; Tue, 9 Oct 2012 21:28:56 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com [98.139.253.104]) by mx1.freebsd.org (Postfix) with ESMTP id 6427D8FC17; Tue, 9 Oct 2012 21:28:56 +0000 (UTC) Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q99LSa8p029723; Tue, 9 Oct 2012 14:28:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1349818118; bh=Q9uhQcHL9xdGAScJWOKzUi3c3KubDdZt9Dpi4Jqdk9c=; h=Subject:From:Reply-To:To:Cc:In-Reply-To:References:Content-Type: Date:Message-ID:Mime-Version:Content-Transfer-Encoding; b=NBsXj8p3WLY5hvaSH81k7XNOYJ+pBvKBEOnubjp7lbHkCasak6/TytnynU6Za6AoY iFI4/tAjI60qRD+OG7M8eSGBuKU4FnnUnYB+FUpoaNGYM73SlYPFtOfGS4Os0vqzbn 29c4WTLTMs/6pM7svTAyJ/5R/g2F/nJzHD2dd3xE= Subject: Re: zfs zlib and net/zlib conflict From: Sean Bruno To: Peter Jeremy In-Reply-To: <20121008205543.GE61955@server.rulingia.com> References: <1349723829.90401.2.camel@powernoodle.corp.yahoo.com> <20121008205543.GE61955@server.rulingia.com> Content-Type: text/plain; charset="UTF-8" Date: Tue, 09 Oct 2012 14:28:36 -0700 Message-ID: <1349818116.35301.9.camel@powernoodle.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 818116004 Cc: "freebsd-fs@freebsd.org" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: "sbruno@freebsd.org" List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2012 21:28:56 -0000 X-List-Received-Date: Tue, 09 Oct 2012 21:28:56 -0000 On Mon, 2012-10-08 at 13:55 -0700, Peter Jeremy wrote: > On 2012-Oct-08 12:17:09 -0700, Sean Bruno wrote: > >Hey, noted that DDB_CTF and ZFS are one cause of zlib conflict > >currently. Not sure how to get around it. > > Yes. I've noticed similar issues. See > http://www.mavetju.org/mail/view_message.php?list=svn-src-all&id=3571352 > I guess the "fastest" solution would be to simply rename functions in the cddl version of zlibc and update the cddl code appropriately? Sean From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 01:43:44 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A4079419 for ; Wed, 10 Oct 2012 01:43:44 +0000 (UTC) (envelope-from spf72sa9@rhythm.ocn.ne.jp) Received: from smtp.rhythm.ocn.ne.jp (rhythm.ocn.ne.jp [122.28.30.178]) by mx1.freebsd.org (Postfix) with ESMTP id 158F38FC0C for ; Wed, 10 Oct 2012 01:43:43 +0000 (UTC) Received: from rhythm.ocn.ne.jp (p5061-ipngn100509fukuokachu.fukuoka.ocn.ne.jp [153.130.238.61]) by smtp.rhythm.ocn.ne.jp (Postfix) with SMTP id 7861B2838 for ; Wed, 10 Oct 2012 10:16:41 +0900 (JST) From: Wanpeng Qian To: freebsd-fs@freebsd.org Subject: Why zfs need to scan whole fs while I attach a HDD to mirror. Date: Wed, 10 Oct 2012 10:16:28 +0900 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: HidemaruMail 5.70 (WinNT,601) Message-Id: <29CDA684DFBE02spf72sa9@rhythm.ocn.ne.jp> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 01:43:44 -0000 Hi guys my zpool is config as follow: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da2 ONLINE 0 0 0 ada2 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 da1 ONLINE 0 0 0 da4 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 da3 ONLINE 0 0 0 ada3 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 ada0 ONLINE 0 0 0 ada1 ONLINE 0 0 0 cache da5 ONLINE 0 0 0 recently, I replace mirror-3, from two 2T to two 3T. first I detach one hdd, and attach new hdd to that mirror. after attach, zfs starting resilvering as expect. but zfs will scan whole fs to resiver new hdd. I think that is not necessery. only copy one disk data to new one is enough. scan the whole fs will spend a lot of time as follow: scan: resilver in progress since Wed Oct 10 07:12:52 2012 1.20T scanned out of 6.48T at 121M/s, 12h39m to go 308G resilvered, 18.60% done does this function correctly? Regards Qian. From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 05:15:39 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F1A52EBF; Wed, 10 Oct 2012 05:15:38 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) by mx1.freebsd.org (Postfix) with ESMTP id 113368FC12; Wed, 10 Oct 2012 05:15:37 +0000 (UTC) Received: by mail-wi0-f178.google.com with SMTP id hr7so110282wib.13 for ; Tue, 09 Oct 2012 22:15:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=P6OPm+u2rzcKF0WdLy9KYRZ8rHZIEiLf/fVDvz7cq0Q=; b=gw2uAw3VEj01l5ntpVwMLmvgfYCoSNiNgbNkhcYqem7On7eOEuVpKbFxluQEfGvySd t6TxFO97j52wQzZ66Pn5Pgm7csLc7NxE43SnsXAvhgLnkASF9hBMIciuQT+cmNZepFzx JCM2iW1jK+j64nzNARpwb1EykeEc/PNYMyRRXVjxjMp7MdqLpjd/0jaRQO2PcP4iD5Oh xjyQN5A8xS0WTmKlFSYCSX58kfu0Llx9PTL+Mbw7M5fKMUeUB4HuYNKVMQrfVbpGplHD T4jX9gRgkUuE4dqZ+VADvisVrpv8xKZRhD0cZIPUih1mrRxEOjkWUVtFrpIUW57JrKLH 1qpA== MIME-Version: 1.0 Received: by 10.216.197.104 with SMTP id s82mr12887252wen.62.1349846136049; Tue, 09 Oct 2012 22:15:36 -0700 (PDT) Received: by 10.223.66.194 with HTTP; Tue, 9 Oct 2012 22:15:35 -0700 (PDT) In-Reply-To: References: <20120829060158.GA38721@x2.osted.lan> <20120831052003.GA91340@x2.osted.lan> <20120905201531.GA54452@x2.osted.lan> <20120917140055.GA9037@x2.osted.lan> <5061F6E9.6030104@omnilan.de> <5062E0DE.70805@omnilan.de> <5065B873.4070509@omnilan.de> Date: Tue, 9 Oct 2012 22:15:35 -0700 Message-ID: Subject: Re: MPSAFE VFS -- List of upcoming actions From: Kevin Oberman To: attilio@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD FS , Harald Schmalzbauer , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 05:15:39 -0000 On Mon, Oct 8, 2012 at 7:57 AM, Attilio Rao wrote: > On Fri, Sep 28, 2012 at 4:47 PM, Harald Schmalzbauer > wrote: >> schrieb Attilio Rao am 28.09.2012 16:18 (localtime): >>> On Wed, Sep 26, 2012 at 12:02 PM, Harald Schmalzbauer >>> wrote: >>>> ... >>> After many people willing to test fuse on STABLE_9, I made this patch >>> that at least compiles there: >>> http://www.freebsd.org/~attilio/fuse_import/fuse_stable9_241030.patch >> >> Thanks a lot! In the meantime I made the original patch compiling. I >> simply looked at the changes which were made around july in the fuse >> project to follow changes in head (checkpath(), vrecycle() and >> vtruncbuf()) and "reverted" them. >> Since I have no idea about the code I modified, I'm happy that you did a >> more qualified patch set :-) >> >>> Of course, I didn't have a chance to test it because I'm also out for >>> vacation right now but please do and report. >> >> Happy holiday!!! If you're by chance arround the Oktoberfest, drop me a >> note, I'll pay you a Ma=C3=9F (or any other drink if you don't like >> =E2=80=9EWiesnbier=E2=80=9C) :-) > > I really hoped to make this year, but no luck :/ > >>>> ... >>>> Some questions: Is this planned to be mfc'd and if so, how can one kno= w? >>> In which sense "how can one know?". We usually specify MFC timeouts in >>> the commit message (not sure if this answers your concerns). >> >> Yep, that's what I wanted to know. So if there's no MFC timeout in the >> log, it's not intended to be MFCd ever I guess. >> >> Thanks a lot! >> World/Kernel compiled fine in the meantime, I'll do some sshfs tests. > > Did you do any test in the end? > > Thanks, > Attilio i have done same testing and it clearly is more stable than the old kmod. At least operations that crashed my system now work. I did see one weird anomaly, though. I had several NTFS file system mounted, one a Windows OS. I also had a GELI encrypted UFS file system mounted. They were both mounted and working. I finished with the data disk and tried to unmount it. I got no error, but it remained mounted. I did not actually try to access it. Figured it would umount when I shut down or end up dirty and I'd have to fsck it. The unmount attempt was using nautilus/gnome-mount. This is not the odd part, though. After the attempt to unmount the UFS device, I could no longer access the Window_OS file system. an ls showed the mount point to be d--------- and an attempt to list files in the directory reported that the socket was not found. So it looks like the attempt to unmount one NTFS FS deleted the socket for the other. This make absolutely no sense to me, but you understand the underlying opertations better than I do. Repeated efforts have failed to re-create the problem. I'm baffled. It is possible that there is no relationship between the two odd things happening at about the same time (NTFS volume lost socket and UFS disk won;t unmount, but reports no errors), but neither has happened since. FWIW, I also see that no device numbers are listed for the fuse devices: /dev/fuse 184319948 165594236 18725712 90% /media/Media /dev/fuse 110636028 82934424 27701604 75% /media/Windows7_OS How does the system distinguish between them? --=20 R. Kevin Oberman, Network Engineer E-mail: kob6558@gmail.com From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 08:37:38 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 32B63384 for ; Wed, 10 Oct 2012 08:37:38 +0000 (UTC) (envelope-from rs@bytecamp.net) Received: from mail.bytecamp.net (mail.bytecamp.net [212.204.60.9]) by mx1.freebsd.org (Postfix) with ESMTP id 73D508FC16 for ; Wed, 10 Oct 2012 08:37:37 +0000 (UTC) Received: (qmail 48066 invoked by uid 89); 10 Oct 2012 10:30:55 +0200 Received: from stella.bytecamp.net (HELO ?212.204.60.37?) (rs%bytecamp.net@212.204.60.37) by mail.bytecamp.net with CAMELLIA256-SHA encrypted SMTP; 10 Oct 2012 10:30:55 +0200 Message-ID: <5075323E.1040009@bytecamp.net> Date: Wed, 10 Oct 2012 10:30:54 +0200 From: Robert Schulze Organization: bytecamp GmbH User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: NFS not responding+panic on full quota'd ZFS Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 08:37:38 -0000 Hi, we are running several webservers on 8.1-RELEASE-p12 which deliver their data from NFS. Ever since we began using ZFS on the fileserver, we always had to track quota usage, because of the annoying "nfs server not responding" error: when a exported filesystem was full and got write access (including unlink), the server just stopped serving. As of this night, things became more complicated. Webservers either crashed in sync with a "sleepable thread owns a non-sleepable lock" panic or after the following: nfs server 10.0.0.XX:/home/www/XX/YY: not responding nfs_getpages: error -484759920 vm_fault: pager read error, pid 9467 (php-cgi) The not responding filesystem was nearly out of quota. Does anybody know, if these issues with quota'd nfs-exported zfs are fixed in 8.3? with kind regards, Robert Schulze From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 10:18:21 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 271EAFE3; Wed, 10 Oct 2012 10:18:21 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) by mx1.freebsd.org (Postfix) with ESMTP id A535A8FC08; Wed, 10 Oct 2012 10:18:19 +0000 (UTC) Received: from server.rulingia.com (c220-239-248-178.belrs5.nsw.optusnet.com.au [220.239.248.178]) by vps.rulingia.com (8.14.5/8.14.5) with ESMTP id q9AAIAxm002529 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 10 Oct 2012 21:18:11 +1100 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.5/8.14.5) with ESMTP id q9AAI4h7045038 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 10 Oct 2012 21:18:05 +1100 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.5/8.14.5/Submit) id q9AAI4PJ045037; Wed, 10 Oct 2012 21:18:04 +1100 (EST) (envelope-from peter) Date: Wed, 10 Oct 2012 21:18:04 +1100 From: Peter Jeremy To: "sbruno@freebsd.org" Subject: Re: zfs zlib and net/zlib conflict Message-ID: <20121010101804.GA42689@server.rulingia.com> References: <1349723829.90401.2.camel@powernoodle.corp.yahoo.com> <20121008205543.GE61955@server.rulingia.com> <1349818116.35301.9.camel@powernoodle.corp.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sdtB3X0nJg68CQEu" Content-Disposition: inline In-Reply-To: <1349818116.35301.9.camel@powernoodle.corp.yahoo.com> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-fs@freebsd.org" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 10:18:21 -0000 --sdtB3X0nJg68CQEu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-Oct-09 14:28:36 -0700, Sean Bruno wrote: >I guess the "fastest" solution would be to simply rename functions in >the cddl version of zlibc and update the cddl code appropriately? I'd agree that's the fastest. The downside is that renaming functions in the CDDL source would make it harder to import updates. If this approach was used, it should probably be done using #defines in a header file (along the lines of libc's namespace.h). In the long term, it would be nice to have all zlib consumers share a single zlib (though this would entail non-trivial effort). --=20 Peter Jeremy --sdtB3X0nJg68CQEu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlB1S1wACgkQ/opHv/APuIeoDgCfYmstCNonRxNXiIyoVfvb+mhT 5gIAoLHWcqBZDNG5DGP4cDjku3TMNE6L =RUJA -----END PGP SIGNATURE----- --sdtB3X0nJg68CQEu-- From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 11:59:32 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 72EC8983 for ; Wed, 10 Oct 2012 11:59:32 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 222118FC2C for ; Wed, 10 Oct 2012 11:59:31 +0000 (UTC) Received: by mail-qa0-f47.google.com with SMTP id i29so4829091qaf.13 for ; Wed, 10 Oct 2012 04:59:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=nDcC+tCapAeWdlkrxbPS43eZhbZ0i/ZtuSvV4zF7j0s=; b=RMuM6Dfy3wRDFjiPjsfAWSIJo3iBI9UfAOtdxlRUBBBTjCJUDBsfZqLbVsjcz8PPLS WOtghqz99JbjiugpFMLKygUzcieGNE1Rr66dC+Eg9BwYWw/03MYsbvF97rA5mIFUhUCH o9ywJXUG6dbIw45F67sNy8lgnlbYishsIYwto1QhUG6tuq69WJpaUIgtsTYyKezMiN+m f+YpRsOiqkNOKr3eDDCHp7EH6+fJegujBCD4bS6gr9gUbClYOIaetDMPjCGzRUKId3Mk 9Xtv4q2O+7tDjRR+YaqdaLfTM/H7Ci7S2CJXc+AZEwV+oDQCgRj2Bj54nmYyML5irEWe fmzg== MIME-Version: 1.0 Received: by 10.49.85.202 with SMTP id j10mr1132223qez.59.1349870370598; Wed, 10 Oct 2012 04:59:30 -0700 (PDT) Received: by 10.49.82.41 with HTTP; Wed, 10 Oct 2012 04:59:30 -0700 (PDT) In-Reply-To: <29CDA684DFBE02spf72sa9@rhythm.ocn.ne.jp> References: <29CDA684DFBE02spf72sa9@rhythm.ocn.ne.jp> Date: Wed, 10 Oct 2012 12:59:30 +0100 Message-ID: Subject: Re: Why zfs need to scan whole fs while I attach a HDD to mirror. From: Tom Evans To: Wanpeng Qian Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 11:59:32 -0000 On Wed, Oct 10, 2012 at 2:16 AM, Wanpeng Qian wrote: > Hi guys > > my zpool is config as follow: > > NAME STATE READ WRITE CKSUM > storage ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > da2 ONLINE 0 0 0 > ada2 ONLINE 0 0 0 > mirror-1 ONLINE 0 0 0 > da1 ONLINE 0 0 0 > da4 ONLINE 0 0 0 > mirror-2 ONLINE 0 0 0 > da3 ONLINE 0 0 0 > ada3 ONLINE 0 0 0 > mirror-3 ONLINE 0 0 0 > ada0 ONLINE 0 0 0 > ada1 ONLINE 0 0 0 > cache > da5 ONLINE 0 0 0 > > recently, I replace mirror-3, from two 2T to two 3T. > > first I detach one hdd, and attach new hdd to that mirror. I think this is your mistake. Detaching a hard drive puts the whole pool in a DEGRADED state, so it thinks it must scrub the whole pool. If instead of "detach/attach" you had done "attach/replace/detach", the pool would never be DEGRADED, and it would not take so long (or not, this is just speculation). Cheers Tom From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 14:16:40 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CA3D1DD2; Wed, 10 Oct 2012 14:16:40 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id EF7A28FC14; Wed, 10 Oct 2012 14:16:39 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id b5so579556lbd.13 for ; Wed, 10 Oct 2012 07:16:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Emw3WDSv3cx24RzP2Vb83jpaECCKgIrAtXUpxGs5fZY=; b=wun0N2XAePYS3hsFXW64YFISVZvxd7G+PS69SHQrKHRxXnmGn0eNmIwrK7S/rQ/wSI YyXb6gkxlTAJCXF8f8b1q2g/+3NL1oHWTBgJEDeLjaohQy1ONibO8Fby9dmv77XRBcso 1BMXjUcNT+jmtDNmXxzGJJZ3G2/9i/CRInksU1tVl3TsHCBk/APsjQsoLBb1tvwZETpY EQuLOpR8v8EHWdAAtWpLcEGLRAbKKi4uRwHFFnbGFJ0fuvOV8Q5/a5ANgUi/JniwwTWY g9tZZ+ZLlCl1N2P/WVx0sP0VoyFWG6Z1In7sgafL3ixybTRzWXIbcV9pwpP09mLfFqyL 0QaA== MIME-Version: 1.0 Received: by 10.112.28.98 with SMTP id a2mr1356473lbh.110.1349878598383; Wed, 10 Oct 2012 07:16:38 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.112.101.234 with HTTP; Wed, 10 Oct 2012 07:16:38 -0700 (PDT) In-Reply-To: References: <20120829060158.GA38721@x2.osted.lan> <20120831052003.GA91340@x2.osted.lan> <20120905201531.GA54452@x2.osted.lan> <20120917140055.GA9037@x2.osted.lan> <5061F6E9.6030104@omnilan.de> <5062E0DE.70805@omnilan.de> <5065B873.4070509@omnilan.de> Date: Wed, 10 Oct 2012 15:16:38 +0100 X-Google-Sender-Auth: OUg9aRI-HLCYpFCJ3ie2WQewPOU Message-ID: Subject: Re: MPSAFE VFS -- List of upcoming actions From: Attilio Rao To: Kevin Oberman Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD FS , Harald Schmalzbauer , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: attilio@FreeBSD.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 14:16:40 -0000 On Wed, Oct 10, 2012 at 6:15 AM, Kevin Oberman wrote: > On Mon, Oct 8, 2012 at 7:57 AM, Attilio Rao wrote: >> On Fri, Sep 28, 2012 at 4:47 PM, Harald Schmalzbauer >> wrote: >>> schrieb Attilio Rao am 28.09.2012 16:18 (localtime): >>>> On Wed, Sep 26, 2012 at 12:02 PM, Harald Schmalzbauer >>>> wrote: >>>>> ... >>>> After many people willing to test fuse on STABLE_9, I made this patch >>>> that at least compiles there: >>>> http://www.freebsd.org/~attilio/fuse_import/fuse_stable9_241030.patch >>> >>> Thanks a lot! In the meantime I made the original patch compiling. I >>> simply looked at the changes which were made around july in the fuse >>> project to follow changes in head (checkpath(), vrecycle() and >>> vtruncbuf()) and "reverted" them. >>> Since I have no idea about the code I modified, I'm happy that you did = a >>> more qualified patch set :-) >>> >>>> Of course, I didn't have a chance to test it because I'm also out for >>>> vacation right now but please do and report. >>> >>> Happy holiday!!! If you're by chance arround the Oktoberfest, drop me a >>> note, I'll pay you a Ma=C3=9F (or any other drink if you don't like >>> =E2=80=9EWiesnbier=E2=80=9C) :-) >> >> I really hoped to make this year, but no luck :/ >> >>>>> ... >>>>> Some questions: Is this planned to be mfc'd and if so, how can one kn= ow? >>>> In which sense "how can one know?". We usually specify MFC timeouts in >>>> the commit message (not sure if this answers your concerns). >>> >>> Yep, that's what I wanted to know. So if there's no MFC timeout in the >>> log, it's not intended to be MFCd ever I guess. >>> >>> Thanks a lot! >>> World/Kernel compiled fine in the meantime, I'll do some sshfs tests. >> >> Did you do any test in the end? >> >> Thanks, >> Attilio > > i have done same testing and it clearly is more stable than the old > kmod. At least operations that crashed my system now work. > > I did see one weird anomaly, though. I had several NTFS file system > mounted, one a Windows OS. I also had a GELI encrypted UFS file system > mounted. They were both mounted and working. I finished with the data > disk and tried to unmount it. I got no error, but it remained mounted. > I did not actually try to access it. Figured it would umount when I > shut down or end up dirty and I'd have to fsck it. The unmount attempt > was using nautilus/gnome-mount. This is not the odd part, though. Kevin, can you please report steps required to reproduce it in high detail (rather than a description), please? This will help in reproducing it and eventually fixing it. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 14:32:39 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A43F9416 for ; Wed, 10 Oct 2012 14:32:39 +0000 (UTC) (envelope-from freebsd@pki2.com) Received: from btw.pki2.com (btw.pki2.com [IPv6:2001:470:a:6fd::2]) by mx1.freebsd.org (Postfix) with ESMTP id 528468FC08 for ; Wed, 10 Oct 2012 14:32:39 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by btw.pki2.com (8.14.5/8.14.5) with ESMTP id q9AEWHDB006576; Wed, 10 Oct 2012 07:32:18 -0700 (PDT) (envelope-from freebsd@pki2.com) Subject: Re: NFS not responding+panic on full quota'd ZFS From: Dennis Glatting To: Robert Schulze In-Reply-To: <5075323E.1040009@bytecamp.net> References: <5075323E.1040009@bytecamp.net> Content-Type: text/plain; charset="ISO-8859-1" Date: Wed, 10 Oct 2012 07:32:17 -0700 Message-ID: <1349879537.58237.4.camel@btw.pki2.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-yoursite-MailScanner-Information: Dennis Glatting X-yoursite-MailScanner-ID: q9AEWHDB006576 X-yoursite-MailScanner: Found to be clean X-MailScanner-From: freebsd@pki2.com Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 14:32:39 -0000 On Wed, 2012-10-10 at 10:30 +0200, Robert Schulze wrote: > Hi, > > we are running several webservers on 8.1-RELEASE-p12 which deliver their > data from NFS. Ever since we began using ZFS on the fileserver, we > always had to track quota usage, because of the annoying "nfs server not > responding" error: when a exported filesystem was full and got write > access (including unlink), the server just stopped serving. > > As of this night, things became more complicated. Webservers either > crashed in sync with a "sleepable thread owns a non-sleepable lock" > panic or after the following: > > nfs server 10.0.0.XX:/home/www/XX/YY: not responding > nfs_getpages: error -484759920 > vm_fault: pager read error, pid 9467 (php-cgi) > > The not responding filesystem was nearly out of quota. > > Does anybody know, if these issues with quota'd nfs-exported zfs are > fixed in 8.3? > Similar problem, maybe. Does the network still work (ping, etc)? Can you login and run commands? Are you running top in another window? If so, what does it say? I have four ZFS servers (similar, not exact), NFS clients (little activity), that hang under heavy local disk and computational load where the processes are stuck in tx->tx (al la top). A reboot is necessary. I gathered equipment for more round of trying to extract debug information but it is looking like I am going to have to strip ZFS, if not FreeBSD, from a production environment. > with kind regards, > Robert Schulze > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 14:51:16 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1D8DDE88; Wed, 10 Oct 2012 14:51:16 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4A2F88FC1E; Wed, 10 Oct 2012 14:51:14 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id b5so616581lbd.13 for ; Wed, 10 Oct 2012 07:51:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=hexGbezS2gz9Af9x/sKm51PE2C2h0XPy2ueNh9v1P44=; b=wMoyIDVO3wKDtkSAJ210qTIQWE37A1y3PUMJubMoV15liwFO7Cvlewb4vxHEoMXCLJ i/tSev8AF/9u+cdXm2t+/f9s+5h2MnEJHZFXuhU3oKIklv392qGzWS83vYY8ZiN8Vitl nOy4Nlvaa/H1VcTyEg2WO2zH/uSm8T6+4tHr1+2iEC2MWR2M77l4mOwe+7Mrk9qdCsJ5 peEe5g012UcEk9s56p9UIzjhsHrbQd/fyCz6WUHiAiABgtHoUS8l+Di705Xqz5+0Qctf Xouo3M2UNjaa5SUWGz+tBsBDg+OlIL5dstbDNNmlQELwZXRnlH7phTg4OzYKTvHHIABU F2mw== MIME-Version: 1.0 Received: by 10.112.82.103 with SMTP id h7mr9404085lby.50.1349880673855; Wed, 10 Oct 2012 07:51:13 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.112.101.234 with HTTP; Wed, 10 Oct 2012 07:51:13 -0700 (PDT) In-Reply-To: References: <20120829060158.GA38721@x2.osted.lan> <20120831052003.GA91340@x2.osted.lan> <20120905201531.GA54452@x2.osted.lan> <20120917140055.GA9037@x2.osted.lan> <5061F6E9.6030104@omnilan.de> <5062E0DE.70805@omnilan.de> <5065B873.4070509@omnilan.de> Date: Wed, 10 Oct 2012 15:51:13 +0100 X-Google-Sender-Auth: csYx4z67h2c9XwvHVUIkd14CvzM Message-ID: Subject: Re: MPSAFE VFS -- List of upcoming actions From: Attilio Rao To: Kevin Oberman Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD FS , Harald Schmalzbauer , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: attilio@FreeBSD.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 14:51:16 -0000 On Wed, Oct 10, 2012 at 6:15 AM, Kevin Oberman wrote: > On Mon, Oct 8, 2012 at 7:57 AM, Attilio Rao wrote: >> On Fri, Sep 28, 2012 at 4:47 PM, Harald Schmalzbauer >> wrote: >>> schrieb Attilio Rao am 28.09.2012 16:18 (localtime): >>>> On Wed, Sep 26, 2012 at 12:02 PM, Harald Schmalzbauer >>>> wrote: >>>>> ... >>>> After many people willing to test fuse on STABLE_9, I made this patch >>>> that at least compiles there: >>>> http://www.freebsd.org/~attilio/fuse_import/fuse_stable9_241030.patch >>> >>> Thanks a lot! In the meantime I made the original patch compiling. I >>> simply looked at the changes which were made around july in the fuse >>> project to follow changes in head (checkpath(), vrecycle() and >>> vtruncbuf()) and "reverted" them. >>> Since I have no idea about the code I modified, I'm happy that you did = a >>> more qualified patch set :-) >>> >>>> Of course, I didn't have a chance to test it because I'm also out for >>>> vacation right now but please do and report. >>> >>> Happy holiday!!! If you're by chance arround the Oktoberfest, drop me a >>> note, I'll pay you a Ma=C3=9F (or any other drink if you don't like >>> =E2=80=9EWiesnbier=E2=80=9C) :-) >> >> I really hoped to make this year, but no luck :/ >> >>>>> ... >>>>> Some questions: Is this planned to be mfc'd and if so, how can one kn= ow? >>>> In which sense "how can one know?". We usually specify MFC timeouts in >>>> the commit message (not sure if this answers your concerns). >>> >>> Yep, that's what I wanted to know. So if there's no MFC timeout in the >>> log, it's not intended to be MFCd ever I guess. >>> >>> Thanks a lot! >>> World/Kernel compiled fine in the meantime, I'll do some sshfs tests. >> >> Did you do any test in the end? >> >> Thanks, >> Attilio > > i have done same testing and it clearly is more stable than the old > kmod. At least operations that crashed my system now work. > > I did see one weird anomaly, though. I had several NTFS file system > mounted, one a Windows OS. I also had a GELI encrypted UFS file system > mounted. They were both mounted and working. I finished with the data > disk and tried to unmount it. I got no error, but it remained mounted. > I did not actually try to access it. Figured it would umount when I > shut down or end up dirty and I'd have to fsck it. The unmount attempt > was using nautilus/gnome-mount. This is not the odd part, though. > > After the attempt to unmount the UFS device, I could no longer access > the Window_OS file system. an ls showed the mount point to be > d--------- and an attempt to list files in the directory reported that > the socket was not found. So it looks like the attempt to unmount one > NTFS FS deleted the socket for the other. > > This make absolutely no sense to me, but you understand the underlying > opertations better than I do. Repeated efforts have failed to > re-create the problem. I'm baffled. It is possible that there is no > relationship between the two odd things happening at about the same > time (NTFS volume lost socket and UFS disk won;t unmount, but reports > no errors), but neither has happened since. > > FWIW, I also see that no device numbers are listed for the fuse devices: > /dev/fuse 184319948 165594236 18725712 90% /media/Media > /dev/fuse 110636028 82934424 27701604 75% /media/Windows7_OS > > How does the system distinguish between them? Sorry, forgot to reply about this and it is due: differently from fuse4bsd version, this one doesn't do device cloning but uses devfs*cdevpriv() infrastructure. So effectively different filedescriptors are handled internally. This is why it requires further changes to the mount_fusefs(8) (because the vfs_mount operation also need further knowledge on the per-filedescriptor handle and it cannot acquire it easilly because it is not a devfs operation). Thanks, Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 16:45:16 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BA80017C for ; Wed, 10 Oct 2012 16:45:16 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 344BB8FC14 for ; Wed, 10 Oct 2012 16:45:15 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id e12so664214lag.13 for ; Wed, 10 Oct 2012 09:45:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=dDJjUYaQ6ZXxhBOjWgo2J/umI0nnh59fS9Mi2TdUu2E=; b=Lf0sMnGAMg4qEfggh148d0eG9h3N7gZQQ4YmXzXdZcVwndj8K5x61OMjGYwFq/zYr4 NbgU74k2y4oJbsnucIlXc76JLHk7XL7BBIbNkZNGR2YB+Lq+Mclwa9S83Tp7O4PfAhe+ 2cNEWvslmzFOr2r+CZiJDbh//frZIjAg3DtggNC/d5gAvOVrlAJzzpDDbGh6BPY0rzb7 rh8FRHs9H/+9SogumTYHEUZA8MKTuv/uD3mwsOs90w/+oa9LAyQYffdMs3ByslvD/WVd Bf/b/CB3ZUzBE8F8PZ4nqCmpJtrrdPjp9T17VadvQQNl2GWXw3oBRi4uCWnirYsnnn6H KvjQ== MIME-Version: 1.0 Received: by 10.152.104.148 with SMTP id ge20mr20298083lab.51.1349887514687; Wed, 10 Oct 2012 09:45:14 -0700 (PDT) Received: by 10.114.23.230 with HTTP; Wed, 10 Oct 2012 09:45:14 -0700 (PDT) In-Reply-To: <074F3CC1-E29F-4552-840F-A38FDDCC7E76@wimsey.us> References: <074F3CC1-E29F-4552-840F-A38FDDCC7E76@wimsey.us> Date: Wed, 10 Oct 2012 09:45:14 -0700 Message-ID: Subject: Re: Deadlock on zfs import From: Freddie Cash To: David Wimsey Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 16:45:16 -0000 If you boot to single-user mode, disable the auto-import of pools (zfs_enable=no in rc.conf), boot to multi-user, start top in one terminal, and then manually import the pool in another window, and watch top out, does it run out of RAM (putting everything into wired)? You're running dedupe with only 4 GB of RAM, with pools over 90% full. My guess is that it's running out of ARC space trying to import the pool and load the DDT (dedupe table). The above test will show that to be the case (wired memory goes to 100% and system locks up). If that's the case, the only solution I've found is to stick more RAM into the box. If the cause of the "can't import due to running out of ARC" is that you were destroying a ZFS filesystem that had dedupe enabled, then you can try upgrading to 9-STABLE and add the "zfs features" patch that was posted here last week. That will run the "zfs destroy" process in the background and allow the pool to import. It worked for us ("zfs destroy" of a 1 TB filesystem locked up our pool; but the background destroy feature let us work around it for importing). -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 16:59:06 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 18230A55 for ; Wed, 10 Oct 2012 16:59:06 +0000 (UTC) (envelope-from david@wimsey.us) Received: from mail-yh0-f54.google.com (mail-yh0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id BBC018FC14 for ; Wed, 10 Oct 2012 16:59:05 +0000 (UTC) Received: by mail-yh0-f54.google.com with SMTP id s35so196632yhf.13 for ; Wed, 10 Oct 2012 09:58:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=U3+is0vy/s+XGEn7QMA4IwdMMyd5CXUcv9lvysjmqz4=; b=Mwhcycg3NzLJjheFRrjDoUrqr+ro8qe6o76ThR0E2w0CtlTfB5sv+r+oxRpsHJlXh2 zR7uP75bOslsuLKwEmkx0uaQ5mNfQPpbST0niMRvHSPbuLCVSXvWz4xriESluY1tyEq3 DVrUMGnPGLbolQosqfUJdMtVZcP5esIMEvKFPLJL8FkSDZmRR7hfqnG7faQggnOxeyde cxceA6/xJT+3w3O9KhjxR77GNnWUIksufNue5wkk5FoH7/dehJa2D5pe4F9Ry+7xpHYV mIhLV81a4k6Dr9iHucFDuyXQFyVqTRH9rTp3GgxP+WBx2bRO1OxSKej4lZdl3blL0uCl J4hg== Received: by 10.236.125.133 with SMTP id z5mr23663962yhh.121.1349888339181; Wed, 10 Oct 2012 09:58:59 -0700 (PDT) Received: from [10.27.1.242] (cpe-107-015-155-065.nc.res.rr.com. [107.15.155.65]) by mx.google.com with ESMTPS id o66sm1643298yhi.19.2012.10.10.09.58.58 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 10 Oct 2012 09:58:58 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: Deadlock on zfs import From: David Wimsey In-Reply-To: Date: Wed, 10 Oct 2012 12:58:57 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <99965AED-8B44-49F7-9EF0-2CD16DB2EEC9@wimsey.us> References: <074F3CC1-E29F-4552-840F-A38FDDCC7E76@wimsey.us> To: Freddie Cash X-Mailer: Apple Mail (2.1499) X-Gm-Message-State: ALoCoQn2HPD86xD2JyTDn376u5pa5oHadNjTX99/GvxEDaEhCgIn5gS9GdODpn78nLl5hu5qu/rw Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 16:59:06 -0000 I should have followed up on my original message. I presumed it was a = memory issue as I know everyone says to use more RAM so I went ahead and = put another 4GB in the machine and imported the pool. The first import = took a few minutes as it chewed through the disks for some reason but = it did import and since then I've exported and reimported with normal = response times and the machine functions just fine now. After all the = reading I did I came to the conclusion that my setup was just not ZFS = worthy with only 4GB since I calculated that I needed about 2GB just for = DDT. In the one instance I had top running it didn't end up at 100% in wired, = but it came fairly close before the consoles stopped updating. If = anyone cares enough about the issue to look into it deeper I can pull = the extra RAM out and do it again to be sure but I'm considering the = problem resolved as an ID10T error on the admin's part (me). The only change to the whole thing I would like to see is some sort of = console indication of the problem. I realize the difficulty in doing = things when the kernel itself is out of memory but it seems like it'd be = possible to reserve a little space for outputting a 'Don't do that = dummy' type of message for people like me. Thanks David Wimsey On Oct 10, 2012, at 12:45 PM, Freddie Cash wrote: > If you boot to single-user mode, disable the auto-import of pools > (zfs_enable=3Dno in rc.conf), boot to multi-user, start top in one > terminal, and then manually import the pool in another window, and > watch top out, does it run out of RAM (putting everything into wired)? >=20 > You're running dedupe with only 4 GB of RAM, with pools over 90% full. > My guess is that it's running out of ARC space trying to import the > pool and load the DDT (dedupe table). The above test will show that > to be the case (wired memory goes to 100% and system locks up). >=20 > If that's the case, the only solution I've found is to stick more RAM > into the box. >=20 > If the cause of the "can't import due to running out of ARC" is that > you were destroying a ZFS filesystem that had dedupe enabled, then you > can try upgrading to 9-STABLE and add the "zfs features" patch that > was posted here last week. That will run the "zfs destroy" process in > the background and allow the pool to import. It worked for us ("zfs > destroy" of a 1 TB filesystem locked up our pool; but the background > destroy feature let us work around it for importing). >=20 > --=20 > Freddie Cash > fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 20:57:08 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 36CD97C0 for ; Wed, 10 Oct 2012 20:57:08 +0000 (UTC) (envelope-from sean@chittenden.org) Received: from mail01.lax1.stackjet.com (mon01.lax1.stackjet.com [174.136.104.178]) by mx1.freebsd.org (Postfix) with ESMTP id 12C8F8FC17 for ; Wed, 10 Oct 2012 20:57:07 +0000 (UTC) Received: from [10.0.20.147] (fw-01.ge-0-9.sfo2.sunstreamnetworks.com [199.101.128.6]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sean@chittenden.org) by mail01.lax1.stackjet.com (Postfix) with ESMTPSA id 4025F3E8D05 for ; Wed, 10 Oct 2012 13:57:01 -0700 (PDT) From: Sean Chittenden Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: ZFS crashing during snapdir lookup for non-existent snapshot... Message-Id: Date: Wed, 10 Oct 2012 13:57:00 -0700 To: freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) X-Mailer: Apple Mail (2.1498) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 20:57:08 -0000 Using a FreeBSD -STABLE build from 2012-09-17, I now have the ability to = crash FreeBSD/ZFS within a few hours of stress testing. It appears as = though there's a locking problem when attempting to interrogate stats on = a ZFS snapshot that doesn't exist any more. I believe the scenario is as = follows: Background: *) `zfs set snapdir=3Dvisible` /was/ set on a data set *) Snapshots were being run once an hour for weeks, long enough for = zabbix to auto-discover the snapshots as valid file systems. *) `zfs inherit snapdir` was recently set (about a week ago), but zabbix = is still attempting to inquire about no snapshots that are no longer = visible or exist. After snapshots were deleted through the normal process of aging, zabbix = is still interrogating the file system attempting to acquire information = about the now deleted snapshots. FreeBSD crashes once every few minutes when zabbix is running and = pulling ZFS information about the now hidden (or most likely deleted) = snapshots. I believe that zabbix is using getfsspec(3) with the now = stale snapshot name in rapid succession and is somehow triggering a race = when there are two concurrent calls to two different non-existent = snapshots. -sc kernel: Fatal trap 12: page fault while in kernel mode kernel: cpuid =3D 0; apic id =3D 00 kernel: fault virtual address =3D 0x368 kernel: fault code =3D supervisor read data, page not = present kernel: instruction pointer =3D 0x20:0xffffffff80922be2 kernel: stack pointer =3D 0x28:0xffffff8487d7b0d0 kernel: frame pointer =3D 0x28:0xffffff8487d7b170 kernel: code segment =3D base 0x0, limit 0xfffff, type 0x1b kernel: =3D DPL 0, pres 1, long 1, def32 0, gran 1 kernel: processor eflags =3D interrupt enabled, resume, IOPL =3D 0 kernel: current process =3D 3536 (zabbix_agentd) kernel: trap number =3D 12 kernel: panic: page fault kernel: cpuid =3D 0 kernel: KDB: stack backtrace: kernel: #0 0xffffffff80950800 at kdb_backtrace+0x60 kernel: #1 0xffffffff8091ac2d at panic+0x1fd kernel: #2 0xffffffff80c21858 at trap_fatal+0x388 kernel: #3 0xffffffff80c21b23 at trap_pfault+0x2b3 kernel: #4 0xffffffff80c212b5 at trap+0x5b5 kernel: #5 0xffffffff80c0ba22 at calltrap+0x8 kernel: #6 0xffffffff8092271e at _sx_xlock+0x5e kernel: #7 0xffffffff816e9384 at zfsctl_snapdir_lookup+0x124 kernel: #8 0xffffffff80cb385f at VOP_LOOKUP_APV+0x5f kernel: #9 0xffffffff809a307f at lookup+0x5ef kernel: #10 0xffffffff809a263d at namei+0x62d kernel: #11 0xffffffff809b2b39 at kern_statfs+0x89 kernel: #12 0xffffffff809b2a80 at sys_statfs+0x20 kernel: #13 0xffffffff80c22134 at amd64_syscall+0x334 FreeBSD example.com 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #1: Mon Sep 17 = 04:34:37 UTC 2012 root@example.com:/usr/obj/usr/src/sys/GENERIC = amd64 0xffffffff80922be2 is in _sx_xlock_hard = (/usr/src/sys/kern/kern_sx.c:546). 541 x =3D sx->sx_lock; 542 if ((sx->lock_object.lo_flags & SX_NOADAPTIVE) = =3D=3D 0) { 543 if ((x & SX_LOCK_SHARED) =3D=3D 0) { 544 x =3D SX_OWNER(x); 545 owner =3D (struct thread *)x; 546 if (TD_IS_RUNNING(owner)) { 547 if = (LOCK_LOG_TEST(&sx->lock_object, 0)) 548 CTR3(KTR_LOCK, 549 "%s: spinning on %p = held by %p", 550 __func__, = sx, owner); -- Sean Chittenden sean@chittenden.org From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 21:09:11 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 53829A25 for ; Wed, 10 Oct 2012 21:09:11 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7FD808FC08 for ; Wed, 10 Oct 2012 21:09:10 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA24084; Thu, 11 Oct 2012 00:08:49 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TM3WL-000LPu-7D; Thu, 11 Oct 2012 00:08:49 +0300 Message-ID: <5075E3E0.7060706@FreeBSD.org> Date: Thu, 11 Oct 2012 00:08:48 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: Sean Chittenden Subject: Re: ZFS crashing during snapdir lookup for non-existent snapshot... References: In-Reply-To: X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 21:09:11 -0000 on 10/10/2012 23:57 Sean Chittenden said the following: > Using a FreeBSD -STABLE build from 2012-09-17, I now have the ability to crash FreeBSD/ZFS within a few hours of stress testing. It appears as though there's a locking problem when attempting to interrogate stats on a ZFS snapshot that doesn't exist any more. I believe the scenario is as follows: > > Background: > > *) `zfs set snapdir=visible` /was/ set on a data set > > *) Snapshots were being run once an hour for weeks, long enough for zabbix to auto-discover the snapshots as valid file systems. > > *) `zfs inherit snapdir` was recently set (about a week ago), but zabbix is still attempting to inquire about no snapshots that are no longer visible or exist. > > > After snapshots were deleted through the normal process of aging, zabbix is still interrogating the file system attempting to acquire information about the now deleted snapshots. > > FreeBSD crashes once every few minutes when zabbix is running and pulling ZFS information about the now hidden (or most likely deleted) snapshots. I believe that zabbix is using getfsspec(3) with the now stale snapshot name in rapid succession and is somehow triggering a race when there are two concurrent calls to two different non-existent snapshots. > > -sc > > > kernel: Fatal trap 12: page fault while in kernel mode > kernel: cpuid = 0; apic id = 00 > kernel: fault virtual address = 0x368 > kernel: fault code = supervisor read data, page not present > kernel: instruction pointer = 0x20:0xffffffff80922be2 > kernel: stack pointer = 0x28:0xffffff8487d7b0d0 > kernel: frame pointer = 0x28:0xffffff8487d7b170 > kernel: code segment = base 0x0, limit 0xfffff, type 0x1b > kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 > kernel: processor eflags = interrupt enabled, resume, IOPL = 0 > kernel: current process = 3536 (zabbix_agentd) > kernel: trap number = 12 > kernel: panic: page fault > kernel: cpuid = 0 > kernel: KDB: stack backtrace: > kernel: #0 0xffffffff80950800 at kdb_backtrace+0x60 > kernel: #1 0xffffffff8091ac2d at panic+0x1fd > kernel: #2 0xffffffff80c21858 at trap_fatal+0x388 > kernel: #3 0xffffffff80c21b23 at trap_pfault+0x2b3 > kernel: #4 0xffffffff80c212b5 at trap+0x5b5 > kernel: #5 0xffffffff80c0ba22 at calltrap+0x8 > kernel: #6 0xffffffff8092271e at _sx_xlock+0x5e > kernel: #7 0xffffffff816e9384 at zfsctl_snapdir_lookup+0x124 > kernel: #8 0xffffffff80cb385f at VOP_LOOKUP_APV+0x5f > kernel: #9 0xffffffff809a307f at lookup+0x5ef > kernel: #10 0xffffffff809a263d at namei+0x62d > kernel: #11 0xffffffff809b2b39 at kern_statfs+0x89 > kernel: #12 0xffffffff809b2a80 at sys_statfs+0x20 > kernel: #13 0xffffffff80c22134 at amd64_syscall+0x334 > > FreeBSD example.com 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #1: Mon Sep 17 04:34:37 UTC 2012 root@example.com:/usr/obj/usr/src/sys/GENERIC amd64 > > 0xffffffff80922be2 is in _sx_xlock_hard (/usr/src/sys/kern/kern_sx.c:546). > 541 x = sx->sx_lock; > 542 if ((sx->lock_object.lo_flags & SX_NOADAPTIVE) == 0) { > 543 if ((x & SX_LOCK_SHARED) == 0) { > 544 x = SX_OWNER(x); > 545 owner = (struct thread *)x; > 546 if (TD_IS_RUNNING(owner)) { > 547 if (LOCK_LOG_TEST(&sx->lock_object, 0)) > 548 CTR3(KTR_LOCK, > 549 "%s: spinning on %p held by %p", > 550 __func__, sx, owner); > Could you please rather list frame #7 (zfsctl_snapdir_lookup+0x124)? -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 22:45:42 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D4419C13; Wed, 10 Oct 2012 22:45:42 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id EDA6F8FC16; Wed, 10 Oct 2012 22:45:41 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id BAA25025; Thu, 11 Oct 2012 01:45:37 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TM521-000LXO-F2; Thu, 11 Oct 2012 01:45:37 +0300 Message-ID: <5075FA8E.10200@FreeBSD.org> Date: Thu, 11 Oct 2012 01:45:34 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: Sean Chittenden , Pawel Jakub Dawidek Subject: Re: ZFS crashing during snapdir lookup for non-existent snapshot... References: <5075E3E0.7060706@FreeBSD.org> <0A6567E7-3BA5-4F27-AEB2-1C00EDE00641@chittenden.org> <5075EDDD.4030008@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-fs@freebsd.org" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 22:45:42 -0000 [restoring mailing list cc] on 11/10/2012 00:58 Sean Chittenden said the following: >>> I don't have a dump from this particular system, only the backtrace from the crash. The system is ZFS only and I only have a ZFS swapdir. :-/ >>> >>> I have the kernel still so I can poke at the code and the compiled kernel (kernel.symbols). ? What are you looking for? -sc >>> >> >> list *zfsctl_snapdir_lookup+0x124 in kgdb > > (kgdb) list *zfsctl_snapdir_lookup+0x124 > 0xffffffff816e9384 is in zfsctl_snapdir_lookup (/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:992). > 987 *direntflags = ED_CASE_CONFLICT; > 988 #endif > 989 } > 990 > 991 mutex_enter(&sdp->sd_lock); > 992 search.se_name = (char *)nm; > 993 if ((sep = avl_find(&sdp->sd_snaps, &search, &where)) != NULL) { > 994 *vpp = sep->se_root; > 995 VN_HOLD(*vpp); > 996 err = traverse(vpp, LK_EXCLUSIVE | LK_RETRY); It seems that the problem is in Solaris-ism that remained in the code. I think that zfsctl_snapdir_inactive should not destroy sdp, that should be a job of vop_reclaim. Otherwise, if the vnode is re-activated its v_data points to freed memory. >>> On Oct 10, 2012, at 14:08 PM, Andriy Gapon wrote: >>> >>>> on 10/10/2012 23:57 Sean Chittenden said the following: >>>>> Using a FreeBSD -STABLE build from 2012-09-17, I now have the ability to crash FreeBSD/ZFS within a few hours of stress testing. It appears as though there's a locking problem when attempting to interrogate stats on a ZFS snapshot that doesn't exist any more. I believe the scenario is as follows: >>>>> >>>>> Background: >>>>> >>>>> *) `zfs set snapdir=visible` /was/ set on a data set >>>>> >>>>> *) Snapshots were being run once an hour for weeks, long enough for zabbix to auto-discover the snapshots as valid file systems. >>>>> >>>>> *) `zfs inherit snapdir` was recently set (about a week ago), but zabbix is still attempting to inquire about no snapshots that are no longer visible or exist. >>>>> >>>>> >>>>> After snapshots were deleted through the normal process of aging, zabbix is still interrogating the file system attempting to acquire information about the now deleted snapshots. >>>>> >>>>> FreeBSD crashes once every few minutes when zabbix is running and pulling ZFS information about the now hidden (or most likely deleted) snapshots. I believe that zabbix is using getfsspec(3) with the now stale snapshot name in rapid succession and is somehow triggering a race when there are two concurrent calls to two different non-existent snapshots. >>>>> >>>>> -sc >>>>> >>>>> >>>>> kernel: Fatal trap 12: page fault while in kernel mode >>>>> kernel: cpuid = 0; apic id = 00 >>>>> kernel: fault virtual address = 0x368 >>>>> kernel: fault code = supervisor read data, page not present >>>>> kernel: instruction pointer = 0x20:0xffffffff80922be2 >>>>> kernel: stack pointer = 0x28:0xffffff8487d7b0d0 >>>>> kernel: frame pointer = 0x28:0xffffff8487d7b170 >>>>> kernel: code segment = base 0x0, limit 0xfffff, type 0x1b >>>>> kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 >>>>> kernel: processor eflags = interrupt enabled, resume, IOPL = 0 >>>>> kernel: current process = 3536 (zabbix_agentd) >>>>> kernel: trap number = 12 >>>>> kernel: panic: page fault >>>>> kernel: cpuid = 0 >>>>> kernel: KDB: stack backtrace: >>>>> kernel: #0 0xffffffff80950800 at kdb_backtrace+0x60 >>>>> kernel: #1 0xffffffff8091ac2d at panic+0x1fd >>>>> kernel: #2 0xffffffff80c21858 at trap_fatal+0x388 >>>>> kernel: #3 0xffffffff80c21b23 at trap_pfault+0x2b3 >>>>> kernel: #4 0xffffffff80c212b5 at trap+0x5b5 >>>>> kernel: #5 0xffffffff80c0ba22 at calltrap+0x8 >>>>> kernel: #6 0xffffffff8092271e at _sx_xlock+0x5e >>>>> kernel: #7 0xffffffff816e9384 at zfsctl_snapdir_lookup+0x124 >>>>> kernel: #8 0xffffffff80cb385f at VOP_LOOKUP_APV+0x5f >>>>> kernel: #9 0xffffffff809a307f at lookup+0x5ef >>>>> kernel: #10 0xffffffff809a263d at namei+0x62d >>>>> kernel: #11 0xffffffff809b2b39 at kern_statfs+0x89 >>>>> kernel: #12 0xffffffff809b2a80 at sys_statfs+0x20 >>>>> kernel: #13 0xffffffff80c22134 at amd64_syscall+0x334 >>>>> >>>>> FreeBSD example.com 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #1: Mon Sep 17 04:34:37 UTC 2012 root@example.com:/usr/obj/usr/src/sys/GENERIC amd64 >>>>> >>>>> 0xffffffff80922be2 is in _sx_xlock_hard (/usr/src/sys/kern/kern_sx.c:546). >>>>> 541 x = sx->sx_lock; >>>>> 542 if ((sx->lock_object.lo_flags & SX_NOADAPTIVE) == 0) { >>>>> 543 if ((x & SX_LOCK_SHARED) == 0) { >>>>> 544 x = SX_OWNER(x); >>>>> 545 owner = (struct thread *)x; >>>>> 546 if (TD_IS_RUNNING(owner)) { >>>>> 547 if (LOCK_LOG_TEST(&sx->lock_object, 0)) >>>>> 548 CTR3(KTR_LOCK, >>>>> 549 "%s: spinning on %p held by %p", >>>>> 550 __func__, sx, owner); >>>>> >>>> >>>> Could you please rather list frame #7 (zfsctl_snapdir_lookup+0x124)? >>>> >>>> -- >>>> Andriy Gapon >>>> >>> >>> >>> >>> >>> -- >>> Sean Chittenden >>> sean@chittenden.org >>> >> >> >> -- >> Andriy Gapon >> > > > > > -- > Sean Chittenden > sean@chittenden.org > -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Thu Oct 11 09:32:57 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 573A69DC; Thu, 11 Oct 2012 09:32:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 63C2E8FC08; Thu, 11 Oct 2012 09:32:55 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA00482; Thu, 11 Oct 2012 12:32:54 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TMF8P-000Opr-Ry; Thu, 11 Oct 2012 12:32:54 +0300 Message-ID: <50769243.2010208@FreeBSD.org> Date: Thu, 11 Oct 2012 12:32:51 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: Pawel Jakub Dawidek Subject: Re: ZFS crashing during snapdir lookup for non-existent snapshot... References: <5075E3E0.7060706@FreeBSD.org> <0A6567E7-3BA5-4F27-AEB2-1C00EDE00641@chittenden.org> <5075EDDD.4030008@FreeBSD.org> <5075FA8E.10200@FreeBSD.org> In-Reply-To: <5075FA8E.10200@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-fs@freebsd.org" , Sean Chittenden X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Oct 2012 09:32:57 -0000 on 11/10/2012 01:45 Andriy Gapon said the following: > > [restoring mailing list cc] > > on 11/10/2012 00:58 Sean Chittenden said the following: >>>> I don't have a dump from this particular system, only the backtrace from the crash. The system is ZFS only and I only have a ZFS swapdir. :-/ >>>> >>>> I have the kernel still so I can poke at the code and the compiled kernel (kernel.symbols). ? What are you looking for? -sc >>>> >>> >>> list *zfsctl_snapdir_lookup+0x124 in kgdb >> >> (kgdb) list *zfsctl_snapdir_lookup+0x124 >> 0xffffffff816e9384 is in zfsctl_snapdir_lookup (/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:992). >> 987 *direntflags = ED_CASE_CONFLICT; >> 988 #endif >> 989 } >> 990 >> 991 mutex_enter(&sdp->sd_lock); >> 992 search.se_name = (char *)nm; >> 993 if ((sep = avl_find(&sdp->sd_snaps, &search, &where)) != NULL) { >> 994 *vpp = sep->se_root; >> 995 VN_HOLD(*vpp); >> 996 err = traverse(vpp, LK_EXCLUSIVE | LK_RETRY); > > It seems that the problem is in Solaris-ism that remained in the code. > I think that zfsctl_snapdir_inactive should not destroy sdp, that should be a > job of vop_reclaim. Otherwise, if the vnode is re-activated its v_data points > to freed memory. > Particularly I have this scenario in mind: - one thread, T1, performs a vput-ish operation which leads to vop_inactive on a current vnode that represents ".zfs/snapshot" - at the same time T2 executes a lookup that goes into zfsctl_root_lookup - let's assume that at some point T1 is at the very start of zfsctl_snapdir_inactive, it holds just a vnode lock - at the same time T2 is in gfs_dir_lookup->gfs_dir_lookup_static and it has gfs_dir_lock - so T2 finds the 'snapshot' static entry in gfsd_static[] - T2 finds the cached vnode and adds a reference - T2 does gfs_dir_unlock and returns the vnode - now T1 proceeds through zfsctl_snapdir_inactive and destroys the v_data (but without clearing the pointer, even) - T2 uses the vnode and gets a crash Possible resolutions: - make vop_inactive a noop and make vop_reclaim call the current inactive methods - check v_usecount in gfs_file_inactive after gfs_dir_lock is obtained and bail out if it is > 0 (somewhat similar to what zfs_zinactive does) - something else? Easy way to reproduce the problem in one way or another - run many of the following in parallel: while true; do ls -l /pool/fs/.zfs/ >/dev/null; done Here is another panic that is a variation of the above scenario. Duplicate gfs_vop_inactive is called after a "harmless" vop_pathconf call (doesn't touch a vnode). In this case the "shares" entry appears to be a random victim: Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x18 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff825fe7dd stack pointer = 0x28:0xffffff80e040b800 frame pointer = 0x28:0xffffff80e040b830 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, IOPL = 0 current process = 712 (ls) trap number = 12 panic: page fault cpuid = 1 curthread: 0xfffffe0003d8a9a0 KDB: stack backtrace: db_trace_self_wrapper() at 0xffffffff802d2bba = db_trace_self_wrapper+0x2a kdb_backtrace() at 0xffffffff805596fa = kdb_backtrace+0x3a panic() at 0xffffffff8051c2a6 = panic+0x266 trap_fatal() at 0xffffffff8070741d = trap_fatal+0x3ad trap_pfault() at 0xffffffff8070756c = trap_pfault+0x12c trap() at 0xffffffff80707d19 = trap+0x4f9 calltrap() at 0xffffffff806ef903 = calltrap+0x8 --- trap 0xc, rip = 0xffffffff825fe7dd, rsp = 0xffffff80e040b800, rbp = 0xffffff80e040b830 --- gfs_vop_inactive() at 0xffffffff825fe7dd = gfs_vop_inactive+0x1d VOP_INACTIVE_APV() at 0xffffffff80782fb4 = VOP_INACTIVE_APV+0x114 vinactive() at 0xffffffff805c84ad = vinactive+0x15d vputx() at 0xffffffff805ca962 = vputx+0x4d2 vput() at 0xffffffff805ca9ce = vput+0xe kern_pathconf() at 0xffffffff805cd44e = kern_pathconf+0x10e sys_lpathconf() at 0xffffffff805cd4aa = sys_lpathconf+0x1a amd64_syscall() at 0xffffffff80706953 = amd64_syscall+0x313 Xfast_syscall() at 0xffffffff806efbe7 = Xfast_syscall+0xf7 -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Fri Oct 12 10:41:40 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3592D78D; Fri, 12 Oct 2012 10:41:40 +0000 (UTC) (envelope-from c.kworr@gmail.com) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0011B8FC0C; Fri, 12 Oct 2012 10:41:39 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id bi1so2937392pad.13 for ; Fri, 12 Oct 2012 03:41:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=mdN8mY9FIYfXWoc8Shr754di3ymogz8nPQDQBLvchvk=; b=Z0w67+Hh2wZULy1Blh210o4oZfAy2weqfDkW5yng57uKGVyg5oKydwyHkSFSoOK/g8 loqolW6vM+uk6zgwbUJpxKyewyWGTbK7bMKQLhaeWsw5uNhvKtKZePTP/6tNilFFWNrr ObwpMhzXTrkHMXO+rz6LcLAjZFmLmTqmecrAwfpT8NkxI+TzgOehOKTgFWBKD9HHIwsV SQC6ULygxzIlEHtlzHngK/+R8/ow8p+bm5IIPuvXvxyqKnwcijbGmS95GPJkp2oQSQE1 Vgy+uBG/iKlPVA6P2g2gxPMdglIj8MrUnQAmQAMp+oxvyJWl6aIXYn4z6KxBXNH5w15r V8yw== Received: by 10.66.77.7 with SMTP id o7mr10284357paw.37.1350038499364; Fri, 12 Oct 2012 03:41:39 -0700 (PDT) Received: from [192.168.1.128] (mau.donbass.com. [92.242.127.250]) by mx.google.com with ESMTPS id sj5sm4268546pbc.30.2012.10.12.03.41.37 (version=SSLv3 cipher=OTHER); Fri, 12 Oct 2012 03:41:38 -0700 (PDT) Message-ID: <5077F3DF.2030108@gmail.com> Date: Fri, 12 Oct 2012 13:41:35 +0300 From: Volodymyr Kostyrko User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:16.0) Gecko/20121011 Thunderbird/16.0 MIME-Version: 1.0 To: Martin Matuska Subject: Re: [CFT] ZFS feature flag support for 9-STABLE References: <506B4508.3030406@FreeBSD.org> In-Reply-To: <506B4508.3030406@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Oct 2012 10:41:40 -0000 02.10.2012 22:48, Martin Matuska wrote: > ZFS feature flag support is ready to be merged to 9-STABLE. > The scheduled merge date is short after 9.1-RELEASE. > > Early adopters can test new features by applying the following patch > (stable/9 r241135): > http://people.freebsd.org/~mm/patches/zfs/9-stable-zfs-features.patch.gz > > Steps to apply to a clean checked-out source: > cd /path/to/src > patch -p0 < /path/to/9-stable-zfs-features.patch Does this suppose to work on i386? On my test machine this gives an instant panic on trying to mount root system. I've already rebuilt my kernel WITH INVARIANTS, WITNESS and DIAGNOSTIC but it still panics even before I can dump core anywhere. I now proceeding to build a tiny virtual machine to capture the backtrace. -- Sphinx of black quartz, judge my vow. From owner-freebsd-fs@FreeBSD.ORG Sat Oct 13 01:34:32 2012 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EEE41A15; Sat, 13 Oct 2012 01:34:32 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [8.8.178.135]) by mx1.freebsd.org (Postfix) with ESMTP id BEFD18FC08; Sat, 13 Oct 2012 01:34:32 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q9D1YWps080490; Sat, 13 Oct 2012 01:34:32 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q9D1YWVf080486; Sat, 13 Oct 2012 01:34:32 GMT (envelope-from linimon) Date: Sat, 13 Oct 2012 01:34:32 GMT Message-Id: <201210130134.q9D1YWVf080486@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-amd64@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/172631: [ufs] mksnap_ffs(8) deadlock with journaling and some daemons X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Oct 2012 01:34:33 -0000 Old Synopsis: mksnap_ffs deadlock with journaling and some daemons New Synopsis: [ufs] mksnap_ffs(8) deadlock with journaling and some daemons Responsible-Changed-From-To: freebsd-amd64->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Sat Oct 13 01:33:17 UTC 2012 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=172631 From owner-freebsd-fs@FreeBSD.ORG Sat Oct 13 13:59:33 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EB414B52 for ; Sat, 13 Oct 2012 13:59:33 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id AC9448FC08 for ; Sat, 13 Oct 2012 13:59:33 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 682F8ED5; Sat, 13 Oct 2012 15:58:19 +0200 (CEST) Date: Sat, 13 Oct 2012 16:00:10 +0200 From: Pawel Jakub Dawidek To: Wanpeng Qian Subject: Re: Why zfs need to scan whole fs while I attach a HDD to mirror. Message-ID: <20121013140010.GD1383@garage.freebsd.pl> References: <29CDA684DFBE02spf72sa9@rhythm.ocn.ne.jp> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mSxgbZZZvrAyzONB" Content-Disposition: inline In-Reply-To: <29CDA684DFBE02spf72sa9@rhythm.ocn.ne.jp> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Oct 2012 13:59:34 -0000 --mSxgbZZZvrAyzONB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 10, 2012 at 10:16:28AM +0900, Wanpeng Qian wrote: > Hi guys >=20 > my zpool is config as follow: >=20 > NAME STATE READ WRITE CKSUM > storage ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > da2 ONLINE 0 0 0 > ada2 ONLINE 0 0 0 > mirror-1 ONLINE 0 0 0 > da1 ONLINE 0 0 0 > da4 ONLINE 0 0 0 > mirror-2 ONLINE 0 0 0 > da3 ONLINE 0 0 0 > ada3 ONLINE 0 0 0 > mirror-3 ONLINE 0 0 0 > ada0 ONLINE 0 0 0 > ada1 ONLINE 0 0 0 > cache > da5 ONLINE 0 0 0 >=20 > recently, I replace mirror-3, from two 2T to two 3T. >=20 > first I detach one hdd, and attach new hdd to that mirror. >=20 > after attach, zfs starting resilvering as expect. > but zfs will scan whole fs to resiver new hdd. >=20 > I think that is not necessery. only copy one disk data to new one > is enough. scan the whole fs will spend a lot of time as follow: >=20 > scan: resilver in progress since Wed Oct 10 07:12:52 2012 > 1.20T scanned out of 6.48T at 121M/s, 12h39m to go > 308G resilvered, 18.60% done >=20 > does this function correctly? ZFS doesn't use separate block-level mirroring. It has to scan entire file system to see which data is stored on mirror-3 exactly. This has pros and cons of course. Huge advantage of this method is that if your pool is not very full, it will take less time to copy only the live data. Block-level mirroring would blindly copy entire disk. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --mSxgbZZZvrAyzONB Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlB5c+kACgkQForvXbEpPzQSTACgrawIDkAtPTvu9eD9qV1VCpOk mnkAn1gSFVRRc2eqfYtaw6XoDRKKbMoI =ixTP -----END PGP SIGNATURE----- --mSxgbZZZvrAyzONB-- From owner-freebsd-fs@FreeBSD.ORG Sat Oct 13 21:43:25 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8463056D; Sat, 13 Oct 2012 21:43:25 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 1C0D58FC0A; Sat, 13 Oct 2012 21:43:24 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4EAJ+LclCDaFvO/2dsb2JhbAA9CIYRuhmCIAEBBAEjSwsFFg4KAgINGQJZBhOHfwamVJF1gSGKLhYKhF6BEgOVa5AugwmBPzw X-IronPort-AV: E=Sophos;i="4.80,581,1344225600"; d="scan'208";a="186279707" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 13 Oct 2012 17:43:18 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 30C5AB4034; Sat, 13 Oct 2012 17:43:18 -0400 (EDT) Date: Sat, 13 Oct 2012 17:43:18 -0400 (EDT) From: Rick Macklem To: Ivan Voras Message-ID: <1979889899.2197790.1350164598183.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: Subject: Re: NFS server bottlenecks MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE7 (Win)/6.0.10_GA_2692) Cc: FS List X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Oct 2012 21:43:25 -0000 Ivan Voras wrote: > On 5 October 2012 14:54, Rick Macklem wrote: > > Yes, the algorithm for UDP is to trim the least recently used > > entries and that LRU list is how the least recently used is done. > > > > Without that, you'd need to do something like timestamp the entries > > and then scan the entire list to find the least recently used ones. > > Hi, > > I'm reading the cache code, and would like to hear from you if my > undestanding of the LRU list is correct: > > * The list is only necessary for the UDP case Yes, since LRU is not a good criteria for selecting cached entries to be replaced (actually trimmed away). > * Entries are added to the lru list only in nfsrc_getudp(), if the > entry is not cached; if it is cached, it is moved to the front > * An entry is also moved to the front in nfsrvd_updatecache() A new entry is put at the head of the list, since it is the most recent. When entries are referenced, they are moved to the head of the list, since they have been used recently. > * nfsrc_trimcache() removes entries from the LRU list, but the > condition is somewhat complex. It trims from the end of the list, since those entries haven't been recently used. > Can you tell me why there is a > RC_REFCNT flag, and also a rc_refcnt field? I.e. wouldn't the field be > enough? The rc_refcnt is only used for NFSv4 entries that reference open/lock state. This is marked by RC_REFCNT and for the case of RC_REFCNT set and rc_refcnt > 0, the entry cannot be thrown away. It will go away when the next open/lock operation in sequence is received. To be honest, since NFSv4 should be using TCP, this should never be set for UDP. The code probably handles the case for UDP, since during early testing UDP was still being used for NFSv4. > * Is it true that the LRU list is essentially completely orthogonal to > the hash tables? Yes. > I.e. there is nothing which specifially requires that > a specific hash table be locked at the same time as the LRU list.? My > goal with all this is to see if the global lru list could be broken > down similarly to the hash entries, to rework the locking for less > contention. It is true that you don't need to hold both locks concurrently, if you have separate locks for the hash buckets for UDP. However, since all threads handling UDP requests will contend on the one lock for the global LRU list, having to also grab a lock for a hash bucket list seems like it will just be additional overhead to me? If, as you proposed, use separate LRU lists for each hash bucket, then how do you know if the least recently used for one hash backet isn't much more recently used than the least recently used for another hash bucket? (The hash code is using xid, which might be about the same for different clients at the same time.) If you were to switch to doing the hashing based on client IP address, then you would get requests from the same client in the same hash bucket. This would be good for selecting least recently used, but poor for spreading the cached entries across the hash list, since at any given point in time, only a few clients will be actively generating RPC requests, I think? If you timestamp the entries as well as move them to the head of the LRU list for a bucket, then you could compare timestamps and decide which ones should be replaced. It would be more work for the trimming code, but if that code isn't being executed too frequently, that could work to minimize contention on the locks as a tradeoff against increased cache storage and more overhead when trimming is being done. I have no idea if this is worth doing? (I'm always surprised that people still prefer UDP to TCP for NFS, but some do seem to.;-) rick ps: I hope you didn't mind me adding the mailing list. I'd like others to be able to comment/read the discussion.