From owner-freebsd-fs@freebsd.org Sun Jan 10 11:49:09 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 656E2A6B7C8 for ; Sun, 10 Jan 2016 11:49:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 566FC1A3A for ; Sun, 10 Jan 2016 11:49:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0ABn8sd047101 for ; Sun, 10 Jan 2016 11:49:09 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 204764] Filesystem deadlock, process in vodead state Date: Sun, 10 Jan 2016 11:49:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: daniel@blodan.se X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2016 11:49:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204764 --- Comment #3 from daniel@blodan.se --- Thanks for the pointers, I'll get back to you with the output as soon as it happens. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 10 15:02:35 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34E54A6B607 for ; Sun, 10 Jan 2016 15:02:35 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id E7B851C7B for ; Sun, 10 Jan 2016 15:02:34 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:kKPE+BaxcsdZGI5opItdRSn/LSx+4OfEezUN459isYplN5qZpcm9bnLW6fgltlLVR4KTs6sC0LqI9fi4EUU7or+/81k6OKRWUBEEjchE1ycBO+WiTXPBEfjxciYhF95DXlI2t1uyMExSBdqsLwaK+i760zceF13FOBZvIaytQ8iJ35rxj7j60qaQSjsLrQL1Wal1IhSyoFeZnegtqqwmFJwMzADUqGBDYeVcyDAgD1uSmxHh+pX4p8Y7oGwD884mouBaXKjQRIhwY71cAS83KHw44daj4RfZQAaF/XdZXH4+nABFDgLe4Ff9RJin4QXgse8o4iiRPoXTRLs3XTmnp/NxTRbjiyMKMhYk927Kh8hojORQqUTy9FRE34fIbdTNZ7JFdaTHcIZCSA== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CmBADFcZJW/61jaINehH+IU7NfgWaGD4FPEgEBAQEBAQEBgQmCLYIOI2gBIgINGQJbBIhBn1OPb49oDAEggQGFVYxygUkFjjaIXY83hEOIXESOCwIpDC+EKCCFKoEIAQEB X-IronPort-AV: E=Sophos;i="5.20,548,1444708800"; d="scan'208";a="260502449" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 10 Jan 2016 10:01:57 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id A779815F55D for ; Sun, 10 Jan 2016 10:01:57 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id iOuJ-OrKi0oe for ; Sun, 10 Jan 2016 10:01:57 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 3A86015F565 for ; Sun, 10 Jan 2016 10:01:57 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id MG01w8KAlPzk for ; Sun, 10 Jan 2016 10:01:57 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 1A2FA15F55D for ; Sun, 10 Jan 2016 10:01:57 -0500 (EST) Date: Sun, 10 Jan 2016 10:01:57 -0500 (EST) From: Rick Macklem To: FreeBSD Filesystems Message-ID: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> Subject: panic ffs_truncate3 (maybe fuse being evil) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: panic ffs_truncate3 (maybe fuse being evil) Thread-Index: PA5DUOP2RagN6MZVMgYY+16xjDN8aw== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2016 15:02:35 -0000 Hi, When fooling around with GlusterFS, I can get this panic intermittently. (I had a couple yesterday.) This happens on a Dec. 5, 2015 head kernel. panic: ffs_truncate3 - backtrace without the numbers (I just scribbled it off the screen) ffs_truncate() ufs_inactive() VOP_INACTIVE_APV() vinactive() vputx() kern_unlinkat() So, at a glance, it seems that either b_dirty.bv_cnt or b_clean.bv_cnt is non-zero. (There is another case for the panic, but I thought it was less likely?) So, I'm wondering if this might be another side effect of r291460, since after that a new vnode isn't completely zero'd out? However, shouldn't bo_dirty.bv_cnt and bo_clean.bv_cnt be zero when a vnode is recycled? Does this make sense or do some fields of v_bufobj need to be zero'd out by getnewvnode()? GlusterFS is using fuse and I suspect that fuse isn't cleaning out the buffers under some circumstance (I already noticed that there isn't any code in its fuse_vnop_reclaim() and I vaguely recall that there are conditions where VOP_INACTIVE() gets skipped, so that VOP_RECLAIM() has to check for anything that would have been done by VOP_INACTIVE() and do it, if it isn't already done.) Anyhow, if others have thoughts on this (or other hunches w.r.t. what could cause this panic(), please let me know. Thanks, rick From owner-freebsd-fs@freebsd.org Sun Jan 10 15:45:26 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CD584A6909C for ; Sun, 10 Jan 2016 15:45:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 67E931D46 for ; Sun, 10 Jan 2016 15:45:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u0AFjJ1P074173 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 10 Jan 2016 17:45:20 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u0AFjJ1P074173 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u0AFjIXQ074160; Sun, 10 Jan 2016 17:45:18 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 10 Jan 2016 17:45:18 +0200 From: Konstantin Belousov To: Rick Macklem Cc: FreeBSD Filesystems Subject: Re: panic ffs_truncate3 (maybe fuse being evil) Message-ID: <20160110154518.GU3625@kib.kiev.ua> References: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2016 15:45:26 -0000 On Sun, Jan 10, 2016 at 10:01:57AM -0500, Rick Macklem wrote: > Hi, > > When fooling around with GlusterFS, I can get this panic intermittently. > (I had a couple yesterday.) This happens on a Dec. 5, 2015 head kernel. > > panic: ffs_truncate3 > - backtrace without the numbers (I just scribbled it off the screen) > ffs_truncate() > ufs_inactive() > VOP_INACTIVE_APV() > vinactive() > vputx() > kern_unlinkat() > > So, at a glance, it seems that either > b_dirty.bv_cnt > or b_clean.bv_cnt > is non-zero. (There is another case for the panic, but I thought it > was less likely?) > > So, I'm wondering if this might be another side effect of r291460, > since after that a new vnode isn't completely zero'd out? > > However, shouldn't bo_dirty.bv_cnt and bo_clean.bv_cnt be zero when > a vnode is recycled? > Does this make sense or do some fields of v_bufobj need to be zero'd > out by getnewvnode()? Look at the _vdrop(). When a vnode is freed to zone, it is asserted that bufobj queues are empty. I very much doubt that it is possible to leak either buffers or counters by reuse. > > GlusterFS is using fuse and I suspect that fuse isn't cleaning out > the buffers under some circumstance (I already noticed that there > isn't any code in its fuse_vnop_reclaim() and I vaguely recall that > there are conditions where VOP_INACTIVE() gets skipped, so that VOP_RECLAIM() > has to check for anything that would have been done by VOP_INACTIVE() > and do it, if it isn't already done.) But even if fuse leaves the buffers around, is it UFS which panics for you ? I would rather worry about dandling pointers and use after free in fuse, which is a known issue with it anyway. I.e. it could be that fuse operates on reclaimed and reused vnode as its own. > > Anyhow, if others have thoughts on this (or other hunches w.r.t. what > could cause this panic(), please let me know. The ffs_truncate3 was deterministically triggered by a bug in ffs_balloc(). The routine allocated buffers for indirect blocks, but if the blocks cannot be allocated, the buffers where left on queue. See r174973, this was fixed very long time ago. From owner-freebsd-fs@freebsd.org Sun Jan 10 17:20:58 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2B41A6B86E for ; Sun, 10 Jan 2016 17:20:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E34531962 for ; Sun, 10 Jan 2016 17:20:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0AHKwtV047285 for ; Sun, 10 Jan 2016 17:20:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Sun, 10 Jan 2016 17:20:58 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2016 17:20:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 Damjan Jovanovic changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #165234|0 |1 is obsolete| | --- Comment #6 from Damjan Jovanovic --- Created attachment 165366 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D165366&action= =3Dedit support for EXT4 sparse files, version 3 New version of the patch with these changes: * Dropped the (int) cast from the second argument in uiomove(). * Curly braces now match the code style of surrounding code. * ep_sparse_ext and ep_ext are now in a union, since only 1 can be valid. * Read ahead for mmap has been implemented, and the performance of reading = from large memory mapped files has improved more than 400 fold, from 251 kB/seco= nd to 106 MB/second :) --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 10 21:00:07 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFEF6A6BACB for ; Sun, 10 Jan 2016 21:00:07 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CEC471762 for ; Sun, 10 Jan 2016 21:00:07 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0AL01TK004728 for ; Sun, 10 Jan 2016 21:00:07 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201601102100.u0AL01TK004728@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention Date: Sun, 10 Jan 2016 21:00:07 +0000 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2016 21:00:08 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f 4 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Sun Jan 10 22:31:05 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68F72A6B62D for ; Sun, 10 Jan 2016 22:31:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 596AF1ED2 for ; Sun, 10 Jan 2016 22:31:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0AMV4Ns083665 for ; Sun, 10 Jan 2016 22:31:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205938] [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Date: Sun, 10 Jan 2016 22:31:05 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2016 22:31:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205938 --- Comment #5 from commit-hook@freebsd.org --- A commit references this bug: Author: pfg Date: Sun Jan 10 22:30:40 UTC 2016 New revision: 293646 URL: https://svnweb.freebsd.org/changeset/base/293646 Log: MFC r293370: ext2fs: reading mmaped file in Ext4 causes panic Always call brelse(path.ep_bp), fixing reading EXT4 files using mmap(). Patch by Damjan Jovanovic. PR: 205938 Changes: _U stable/10/ stable/10/sys/fs/ext2fs/ext2_bmap.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 10 22:32:07 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 74F8FA6B78A for ; Sun, 10 Jan 2016 22:32:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 658161275 for ; Sun, 10 Jan 2016 22:32:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0AMW6DS088876 for ; Sun, 10 Jan 2016 22:32:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205938] [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Date: Sun, 10 Jan 2016 22:32:07 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2016 22:32:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205938 --- Comment #6 from commit-hook@freebsd.org --- A commit references this bug: Author: pfg Date: Sun Jan 10 22:31:39 UTC 2016 New revision: 293647 URL: https://svnweb.freebsd.org/changeset/base/293647 Log: MFC r293370: ext2fs: reading mmaped file in Ext4 causes panic Always call brelse(path.ep_bp), fixing reading EXT4 files using mmap(). Patch by Damjan Jovanovic. PR: 205938 Changes: _U stable/9/sys/ _U stable/9/sys/fs/ stable/9/sys/fs/ext2fs/ext2_bmap.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 10 22:35:44 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C2945A6B935 for ; Sun, 10 Jan 2016 22:35:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2FF6155F for ; Sun, 10 Jan 2016 22:35:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0AMZiDs093695 for ; Sun, 10 Jan 2016 22:35:44 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205938] [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Date: Sun, 10 Jan 2016 22:35:44 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2016 22:35:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205938 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|In Progress |Closed --- Comment #7 from Pedro F. Giffuni --- MFC'd a little early since avoiding panics is good. Thanks! --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 10 22:49:09 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79ECCA6BC5A for ; Sun, 10 Jan 2016 22:49:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A9E61FDE for ; Sun, 10 Jan 2016 22:49:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0AMn8e0015981 for ; Sun, 10 Jan 2016 22:49:09 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Sun, 10 Jan 2016 22:49:09 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2016 22:49:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 --- Comment #7 from Pedro F. Giffuni --- (In reply to Damjan Jovanovic from comment #6) Nice! The patch looks good but there is something wrong when I apply it: % svn patch --dry-run --strip 1 patch-PR165234 C sys/fs/ext2fs/ext2_bmap.c > rejected hunk @@ -112,9 +109,11 @@ > rejected hunk @@ -124,6 +123,9 @@ C sys/fs/ext2fs/ext2_extents.c > rejected hunk @@ -82,8 +82,7 @@ > rejected hunk @@ -92,8 +91,7 @@ C sys/fs/ext2fs/ext2_extents.h > rejected hunk @@ -85,8 +85,10 @@ C sys/fs/ext2fs/ext2_vnops.c > rejected hunk @@ -1863,16 +1863,13 @@ Summary of conflicts: Text conflicts: 4 --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 10 23:20:03 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6D72A6A869 for ; Sun, 10 Jan 2016 23:20:03 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 72D2D1EB1 for ; Sun, 10 Jan 2016 23:20:02 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:mjh4URe5aLT0wJm3qy/ZxrdQlGMj4u6mDksu8pMizoh2WeGdxc6/YB7h7PlgxGXEQZ/co6odzbGG7ea4ASQp2tWojjMrSNR0TRgLiMEbzUQLIfWuLgnFFsPsdDEwB89YVVVorDmROElRH9viNRWJ+iXhpQAbFhi3DwdpPOO9QteU1JTpkbjqs7ToICx2xxOFKYtoKxu3qQiD/uI3uqBFbpgL9x3Sv3FTcP5Xz247bXianhL7+9vitMU7q3cY6Lod8JtjVqPhY60+Ub1eRB4rN2co/8r1/U3AShCT53gWX2E+nR9BAgyD5xb/CMTfqCz/49B80yrSGMT9TrQ5XHz29aJiQxzshSIvKjk27WzTksw2h6sN80HpnAB234OBONLdD/F5ZK6IJd4= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CqBAAK5pJW/61jaINehH+IU7VEhg8CgU8QAQEBAQEBAQGBCYItgggBAQQjVhACAQgOCgICDRkCAlcCBIhBrxGPXwEBAQEBBQEBAQEBAR2BAYVVhH+Hc4FJBY42iF2PN4RDiFxEjgsCOSuEKCCFMoEIAQEB X-IronPort-AV: E=Sophos;i="5.20,550,1444708800"; d="scan'208";a="262106178" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 10 Jan 2016 18:19:56 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 106BA15F55D; Sun, 10 Jan 2016 18:19:56 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id GqHbO14UdJVT; Sun, 10 Jan 2016 18:19:55 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 67E0215F565; Sun, 10 Jan 2016 18:19:55 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id aOyxgddVbpmQ; Sun, 10 Jan 2016 18:19:55 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 4B88015F55D; Sun, 10 Jan 2016 18:19:55 -0500 (EST) Date: Sun, 10 Jan 2016 18:19:55 -0500 (EST) From: Rick Macklem To: Konstantin Belousov Cc: FreeBSD Filesystems Message-ID: <700310221.155153995.1452467995252.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20160110154518.GU3625@kib.kiev.ua> References: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> <20160110154518.GU3625@kib.kiev.ua> Subject: Re: panic ffs_truncate3 (maybe fuse being evil) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: panic ffs_truncate3 (maybe fuse being evil) Thread-Index: i5Ow7DLuCEiHdCTEDzqw6FyLZAae9A== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2016 23:20:03 -0000 Kostik wrote: > On Sun, Jan 10, 2016 at 10:01:57AM -0500, Rick Macklem wrote: > > Hi, > > > > When fooling around with GlusterFS, I can get this panic intermittently. > > (I had a couple yesterday.) This happens on a Dec. 5, 2015 head kernel. > > > > panic: ffs_truncate3 > > - backtrace without the numbers (I just scribbled it off the screen) > > ffs_truncate() > > ufs_inactive() > > VOP_INACTIVE_APV() > > vinactive() > > vputx() > > kern_unlinkat() > > > > So, at a glance, it seems that either > > b_dirty.bv_cnt > > or b_clean.bv_cnt > > is non-zero. (There is another case for the panic, but I thought it > > was less likely?) > > > > So, I'm wondering if this might be another side effect of r291460, > > since after that a new vnode isn't completely zero'd out? > > > > However, shouldn't bo_dirty.bv_cnt and bo_clean.bv_cnt be zero when > > a vnode is recycled? > > Does this make sense or do some fields of v_bufobj need to be zero'd > > out by getnewvnode()? > Look at the _vdrop(). When a vnode is freed to zone, it is asserted > that bufobj queues are empty. I very much doubt that it is possible > to leak either buffers or counters by reuse. > Ok. I'll take a look but, yes, it doesn't sound like the fields could be left bogus when the vnode gets recycled. > > > > GlusterFS is using fuse and I suspect that fuse isn't cleaning out > > the buffers under some circumstance (I already noticed that there > > isn't any code in its fuse_vnop_reclaim() and I vaguely recall that > > there are conditions where VOP_INACTIVE() gets skipped, so that > > VOP_RECLAIM() > > has to check for anything that would have been done by VOP_INACTIVE() > > and do it, if it isn't already done.) > But even if fuse leaves the buffers around, is it UFS which panics for > you ? I would rather worry about dandling pointers and use after free in > fuse, which is a known issue with it anyway. I.e. it could be that fuse > operates on reclaimed and reused vnode as its own. > > > > > Anyhow, if others have thoughts on this (or other hunches w.r.t. what > > could cause this panic(), please let me know. > > The ffs_truncate3 was deterministically triggered by a bug in ffs_balloc(). > The routine allocated buffers for indirect blocks, but if the blocks cannot > be allocated, the buffers where left on queue. See r174973, this was fixed > very long time ago. > But this was a one month old kernel (around r291900, although I don't know the exact r#, but it was Dec. 5, 2015), so it definitely has this fix in it. When I see it again, I will try and see what the v_bufobj fields look like. Thanks, rick From owner-freebsd-fs@freebsd.org Mon Jan 11 02:05:30 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BD60A6B0B8 for ; Mon, 11 Jan 2016 02:05:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3D3AF1D32 for ; Mon, 11 Jan 2016 02:05:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0B25URs048393 for ; Mon, 11 Jan 2016 02:05:30 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Mon, 11 Jan 2016 02:05:30 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 02:05:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 --- Comment #8 from Damjan Jovanovic --- (In reply to Pedro F. Giffuni from comment #7) Thank you. It applies here on 2 computers. Are you applying it to the latest CURRENT, and have no earlier version of that patch already applied? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 02:28:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00A15A6B8C4 for ; Mon, 11 Jan 2016 02:28:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E26631870 for ; Mon, 11 Jan 2016 02:28:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0B2SeET064508 for ; Mon, 11 Jan 2016 02:28:40 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Mon, 11 Jan 2016 02:28:41 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 02:28:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 --- Comment #9 from Pedro F. Giffuni --- (In reply to Damjan Jovanovic from comment #8) Hi. Yes, I am using current, checked out from subversion, and I tried with: - svn patch - patch from base - gpatch They all fail :(. Try to email me the patch directly to my FreeBSD address. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 11:18:01 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95D7CA6B7FF for ; Mon, 11 Jan 2016 11:18:01 +0000 (UTC) (envelope-from octavianh@gmail.com) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A0261A20 for ; Mon, 11 Jan 2016 11:18:01 +0000 (UTC) (envelope-from octavianh@gmail.com) Received: by mail-qg0-x22a.google.com with SMTP id b35so269715806qge.0 for ; Mon, 11 Jan 2016 03:18:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=rILZ2mg8JdMNHWGoLdZGyok329CfK+vWGsrzo94u2UU=; b=Hjczu7KAFaMXrEmD3/SC0vjSqmYYx/3cyL1zUd8gyTak94BZHEo59dfXymG8YTu0vn kTCtkuFo/Hbi5jK4reLww4/ZKXUKdiIhBllUSEcDneoOqANgTjjuZ0ZquHCaKhtKJrWR TJVdnl7TccKPwHAhWdiBr/m97JOo98swcJ51tDucNw/fWVRwITnY/Jtrm1b+C/eC/yal Duf8xve3p1UcY5gG/nRGwUavbymhUYHe5omNi6SJLPqRoDIACW1bVnbUyafbFFhZ+OI/ 6WVhwI3kdzaxNSOh9PqsAaiyuyMsGCHgigYpsftUholKGEGSvVUOqsWRKDqL09xrnGN1 /SOQ== MIME-Version: 1.0 X-Received: by 10.140.220.207 with SMTP id q198mr34346158qhb.24.1452511080449; Mon, 11 Jan 2016 03:18:00 -0800 (PST) Received: by 10.140.81.16 with HTTP; Mon, 11 Jan 2016 03:18:00 -0800 (PST) Received: by 10.140.81.16 with HTTP; Mon, 11 Jan 2016 03:18:00 -0800 (PST) In-Reply-To: References: Date: Mon, 11 Jan 2016 03:18:00 -0800 Message-ID: Subject: Question on gmirror and zfs fs behavior in unusual setup From: Octavian Hornoiu To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 11:18:01 -0000 I currently have several storage servers. For historical reasons they have 6x 1TB Western Digital Black SATA drives in each server. Configuration is as follows: GPT disk config with boot sector /dev/ada0p1 freebsd-boot 64k /dev/ada0p2 freebsd-swap 1G /dev/ada0p3 freebsd-ufs 30G /dev/ada0p4 freebsd-zfs rest of drive The drive names are ada0 through ada5. The six drives all have the same partition scheme. - They are all bootable - Each swap has a label from swap0 through swap5 which all mount on boot - The UFS partitions are all in mirror/rootfs mirrored using gmirror in a 6 way mirror (The goal of the boot and mirror redundancy is any drive can die and I can still boot off any other drive like nothing happened. This partition contains the entire OS. - The zfs partitions are in RAIDZ-2 configuration and are redundant automatically. They contain the network accessible storage data. My dilemma is this. I am upgrading to 5 TB Western Digital Black drives. I have replaced drive ada5 as a test. I used the -a 4k command while partitioning to make sure sector alignment is correct. There are two major changes: - ada5p3 is now 100 G - ada5p4 is now much larger due to the size of the drive My understanding is that zfs will automatically change the total volume size once all drives are upgraded to the new 5 TB drives. Please correct me if I'm wrong! The resilver went without a hitch. My concern is with gmirror. Will gmirror grow to fit the new 100 G size automatically once the last drive is replaced? I got no errors using insert with the 100 G partition into the mix with the other 5 30 G partitions. It synchronized fine. The volume shows as complete and all providers are healthy. Anyone with knowledge of gmirror and zfs replication able to confirm that they'll grow automatically once all 6 drives are replaced or do I have to sync them at existing size and do some growfs trick later? Thanks! From owner-freebsd-fs@freebsd.org Mon Jan 11 11:45:05 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 707B7A6C0AF for ; Mon, 11 Jan 2016 11:45:05 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x232.google.com (mail-wm0-x232.google.com [IPv6:2a00:1450:400c:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0AA0514C1 for ; Mon, 11 Jan 2016 11:45:04 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x232.google.com with SMTP id f206so209261694wmf.0 for ; Mon, 11 Jan 2016 03:45:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-type:content-transfer-encoding; bh=p9fn3gIGhovpXm0N3YBHXn5HvSd9LEW2+2YxQ0RBI3o=; b=npqWY38HgH2WbXupErZBG6IVqmcNnmy/ivfcE1o6xObfzT2gOcij+pOSerWjCYxTLV tCT2SxdSOYamAM1IfCl0IClEM0uRgyz8oQz40O78TuHCWIxvoGTY/pjovhKcRVJqCtKG FT4bN9Oa0WfN/TjNycwY0EtJs0D2V0G6dlI0THiQrD9z1xEAzqBUhPYo/o7/LdczHQtM CnBb874hoKuLT6KpykGzBollzfiLUtQVG6GaeTs3NeDvR5AHsbbkHOOlBbZYtlsZXoji TdD4Vpa/g/F3mDIyD8BwKozuAUdMqnEHgHox4aEPkfK7h0O53CT4GUtxQuiRJkLP5zk0 WKDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=p9fn3gIGhovpXm0N3YBHXn5HvSd9LEW2+2YxQ0RBI3o=; b=ivhSX74+MG3ot54YAyIUBSl69k473knTro0Ht1qzijgtKv6GamvLlEprhQGq4SCWbI /X/mcH4mgFP3nCc3+OoDtNyT6DHzu4DfsXwD8Y4LusOmvDETed3dRBMN1nsoR5al4E3F CTy9IlKcn+qbeL5YN959FT8QKzyOB9FEs7Un7a7cKnmxiy5imQSa1NjdkPYr8Ehta5rM 9B6nv7nC4cCqfPp61SgZpt9drJsTeLBNi+nF2Gm0PXN3SGfBEjx8eJIDd1tJuEdMjDuE rSIysTS3xRavL1NKhUBaLCe6EwFlmaBU0p9sevKgvnkTl753FSg6WUOMXLLxXPeofw7I aBJA== X-Gm-Message-State: ALoCoQklbI6nRCsf2Tp0Py/loYjbeaTLVVbgqLqpzKpJ2zBzhapqtz2aSZwHbI+4GdL0+nPWMOcC0QmYhiky+Ib4m5ZGMoM04w== X-Received: by 10.28.170.139 with SMTP id t133mr13827445wme.50.1452512702025; Mon, 11 Jan 2016 03:45:02 -0800 (PST) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id bg10sm119080370wjb.46.2016.01.11.03.45.00 for (version=TLSv1/SSLv3 cipher=OTHER); Mon, 11 Jan 2016 03:45:00 -0800 (PST) Subject: Re: Question on gmirror and zfs fs behavior in unusual setup To: freebsd-fs@freebsd.org References: From: Steven Hartland Message-ID: <569395CC.6060104@multiplay.co.uk> Date: Mon, 11 Jan 2016 11:45:16 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 11:45:05 -0000 On 11/01/2016 11:18, Octavian Hornoiu wrote: > I currently have several storage servers. For historical reasons they have > 6x 1TB Western Digital Black SATA drives in each server. Configuration is > as follows: > > GPT disk config with boot sector > /dev/ada0p1 freebsd-boot 64k > /dev/ada0p2 freebsd-swap 1G > /dev/ada0p3 freebsd-ufs 30G > /dev/ada0p4 freebsd-zfs rest of drive > > The drive names are ada0 through ada5. > > The six drives all have the same partition scheme. > - They are all bootable > - Each swap has a label from swap0 through swap5 which all mount on boot > - The UFS partitions are all in mirror/rootfs mirrored using gmirror in a 6 > way mirror (The goal of the boot and mirror redundancy is any drive can die > and I can still boot off any other drive like nothing happened. This > partition contains the entire OS. > - The zfs partitions are in RAIDZ-2 configuration and are redundant > automatically. They contain the network accessible storage data. > > My dilemma is this. I am upgrading to 5 TB Western Digital Black drives. I > have replaced drive ada5 as a test. I used the -a 4k command while > partitioning to make sure sector alignment is correct. There are two major > changes: > > - ada5p3 is now 100 G > - ada5p4 is now much larger due to the size of the drive > > My understanding is that zfs will automatically change the total volume > size once all drives are upgraded to the new 5 TB drives. Please correct me > if I'm wrong! The resilver went without a hitch. Correct you just need to ensure that autoexpand is enabled on the pool e.g. zpool set autoexpand=on tank > My concern is with gmirror. Will gmirror grow to fit the new 100 G size > automatically once the last drive is replaced? I got no errors using insert > with the 100 G partition into the mix with the other 5 30 G partitions. It > synchronized fine. The volume shows as complete and all providers are > healthy. I'm not sure with gmirror 100% but the following seems to detail what you want: https://lists.freebsd.org/pipermail/freebsd-questions/2007-August/156466.html > > Anyone with knowledge of gmirror and zfs replication able to confirm that > they'll grow automatically once all 6 drives are replaced or do I have to > sync them at existing size and do some growfs trick later? > > Thanks! > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Mon Jan 11 11:59:03 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 232A9A6C536 for ; Mon, 11 Jan 2016 11:59:03 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD25F1BD0 for ; Mon, 11 Jan 2016 11:59:02 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 652CA28431; Mon, 11 Jan 2016 12:49:20 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 5B5202842B; Mon, 11 Jan 2016 12:49:19 +0100 (CET) Message-ID: <569396BF.90801@quip.cz> Date: Mon, 11 Jan 2016 12:49:19 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: Octavian Hornoiu , freebsd-fs@freebsd.org Subject: Re: Question on gmirror and zfs fs behavior in unusual setup References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 11:59:03 -0000 Octavian Hornoiu wrote on 01/11/2016 12:18: > I currently have several storage servers. For historical reasons they have > 6x 1TB Western Digital Black SATA drives in each server. Configuration is > as follows: > > GPT disk config with boot sector > /dev/ada0p1 freebsd-boot 64k > /dev/ada0p2 freebsd-swap 1G > /dev/ada0p3 freebsd-ufs 30G > /dev/ada0p4 freebsd-zfs rest of drive > > The drive names are ada0 through ada5. > > The six drives all have the same partition scheme. > - They are all bootable > - Each swap has a label from swap0 through swap5 which all mount on boot > - The UFS partitions are all in mirror/rootfs mirrored using gmirror in a 6 > way mirror (The goal of the boot and mirror redundancy is any drive can die > and I can still boot off any other drive like nothing happened. This > partition contains the entire OS. > - The zfs partitions are in RAIDZ-2 configuration and are redundant > automatically. They contain the network accessible storage data. > > My dilemma is this. I am upgrading to 5 TB Western Digital Black drives. I > have replaced drive ada5 as a test. I used the -a 4k command while > partitioning to make sure sector alignment is correct. There are two major > changes: > > - ada5p3 is now 100 G > - ada5p4 is now much larger due to the size of the drive > > My understanding is that zfs will automatically change the total volume > size once all drives are upgraded to the new 5 TB drives. Please correct me > if I'm wrong! The resilver went without a hitch. > > My concern is with gmirror. Will gmirror grow to fit the new 100 G size > automatically once the last drive is replaced? I got no errors using insert > with the 100 G partition into the mix with the other 5 30 G partitions. It > synchronized fine. The volume shows as complete and all providers are > healthy. No gmirror will not expand automatically nor manually. You will have 30GB mirror and 70GB of unused space. If you really need to expand this mirror, you need to create 100GB partition, format with uFS, copy data from old mirror, destroy old mirror and create new bigger from this 100GB partition. > Anyone with knowledge of gmirror and zfs replication able to confirm that > they'll grow automatically once all 6 drives are replaced or do I have to > sync them at existing size and do some growfs trick later? If your pool has autoexpand=on. Maybe reboot or zpool offline / online is needed. I don't remember it well. Miroslav Lachman From owner-freebsd-fs@freebsd.org Mon Jan 11 12:08:14 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65907A6CE76 for ; Mon, 11 Jan 2016 12:08:14 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-outbound.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EFD8C1291 for ; Mon, 11 Jan 2016 12:08:13 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) by smtp-outbound.userve.net (8.15.1/8.15.1) with ESMTPS id u0BC7ePU091874 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 11 Jan 2016 12:07:40 GMT (envelope-from matt.churchyard@userve.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1452514065; bh=Y8D/HI9cxty1CeO/dbSdUrONvnoIvuL1er/xmp1xeKA=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=av8inmVeE/9dby1hJgmC6CALPRjkG38Z02FtMud78xxhlXhhUExo0D7CsYPXuQpzg 9m2cblAEI34hHvgKWpkcJ6KPZGXC1+dmRpoO7333akok8BrvKCKpWt921vmcBsD/Zt l/wnDy4fJE9HhKa9rQh8ggjeDg9tI2o3uNtwEc+w= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Mon, 11 Jan 2016 12:07:34 +0000 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Mon, 11 Jan 2016 12:07:34 +0000 From: Matt Churchyard To: Octavian Hornoiu CC: freebsd-fs Subject: RE: Question on gmirror and zfs fs behavior in unusual setup Thread-Topic: Question on gmirror and zfs fs behavior in unusual setup Thread-Index: AQHRTGOVYc8FhRGcRU28LzQfm5XmY572NkMQ Date: Mon, 11 Jan 2016 12:07:34 +0000 Message-ID: <9522d5cccd704b8fbe6cfe00d3bbd51a@SERVER.ad.usd-group.com> References: In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 12:08:14 -0000 >I currently have several storage servers. For historical reasons they have= 6x 1TB Western Digital Black SATA drives in each server. Configuration is = >as follows: >GPT disk config with boot sector >/dev/ada0p1 freebsd-boot 64k >/dev/ada0p2 freebsd-swap 1G >/dev/ada0p3 freebsd-ufs 30G >/dev/ada0p4 freebsd-zfs rest of drive >The drive names are ada0 through ada5. >The six drives all have the same partition scheme. >- They are all bootable >- Each swap has a label from swap0 through swap5 which all mount on boot >- The UFS partitions are all in mirror/rootfs mirrored using gmirror in a = 6 way mirror (The goal of the boot and mirror redundancy is any drive can >= die and I can still boot off any other drive like nothing happened. This pa= rtition contains the entire OS. >- The zfs partitions are in RAIDZ-2 configuration and are redundant automa= tically. They contain the network accessible storage data. >My dilemma is this. I am upgrading to 5 TB Western Digital Black drives. I= have replaced drive ada5 as a test. I used the -a 4k command while >partit= ioning to make sure sector alignment is correct. There are two major >changes: >- ada5p3 is now 100 G >- ada5p4 is now much larger due to the size of the drive >My understanding is that zfs will automatically change the total volume si= ze once all drives are upgraded to the new 5 TB drives. Please correct >me = if I'm wrong! The resilver went without a hitch. You may have to run "zpool online -e pool" once all the disk have been repl= aced, but yes it should be fairly easy to get ZFS to pick up the new space. The only other issue you may see is that if you built the original pool wit= h 512b sectors (ashift 9) you may find "zpool status" start complaining tha= t you are configured for 512b sectors when your disks are 4k (I haven't che= cked but considering the size I expect those 5TB disks are 4k). If that hap= pens you either have to live with the warning or rebuild the pool. >My concern is with gmirror. Will gmirror grow to fit the new 100 G size au= tomatically once the last drive is replaced? I got no errors using insert >= with the 100 G partition into the mix with the other 5 30 G partitions. It = synchronized fine. The volume shows as complete and all providers are >heal= thy. A quick test suggests you'll need to run "gmirror resize provider" once all= the disks are replaced to get gmirror to update the size stored in the met= adata -=20 # gmirror list Geom name: test State: COMPLETE Components: 2 ... Providers: 1. Name: mirror/test Mediasize: 104857088 (100M) Sectorsize: 512 Mode: r0w0e0 Consumers: 1. Name: md0 Mediasize: 209715200 (200M) ... # gmirror resize test # gmirror list ... Providers: 1. Name: mirror/test Mediasize: 209714688 (200M) Sectorsize: 512 Mode: r0w0e0 ... You will then need to expand the filesystem to fill the space using growfs.= Never done this but it should be a fairly straight forward process from wh= at I can see, although it seems resizing while mounted only works on 10.0+ >Anyone with knowledge of gmirror and zfs replication able to confirm that = they'll grow automatically once all 6 drives are replaced or do I have >to = sync them at existing size and do some growfs trick later? >Thanks! From owner-freebsd-fs@freebsd.org Mon Jan 11 12:16:15 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EEB0BA6B251 for ; Mon, 11 Jan 2016 12:16:14 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2FF018C8 for ; Mon, 11 Jan 2016 12:16:14 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 734F428423; Mon, 11 Jan 2016 13:16:12 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 71E5B28431; Mon, 11 Jan 2016 13:16:11 +0100 (CET) Message-ID: <56939D0B.6010509@quip.cz> Date: Mon, 11 Jan 2016 13:16:11 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: Matt Churchyard , Octavian Hornoiu CC: freebsd-fs Subject: Re: Question on gmirror and zfs fs behavior in unusual setup References: <9522d5cccd704b8fbe6cfe00d3bbd51a@SERVER.ad.usd-group.com> In-Reply-To: <9522d5cccd704b8fbe6cfe00d3bbd51a@SERVER.ad.usd-group.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 12:16:15 -0000 Matt Churchyard via freebsd-fs wrote on 01/11/2016 13:07: >> I currently have several storage servers. For historical reasons they have 6x 1TB Western Digital Black SATA drives in each server. Configuration is >as follows: > >> GPT disk config with boot sector >> /dev/ada0p1 freebsd-boot 64k >> /dev/ada0p2 freebsd-swap 1G >> /dev/ada0p3 freebsd-ufs 30G >> /dev/ada0p4 freebsd-zfs rest of drive > >> The drive names are ada0 through ada5. > >> The six drives all have the same partition scheme. >> - They are all bootable >> - Each swap has a label from swap0 through swap5 which all mount on boot >> - The UFS partitions are all in mirror/rootfs mirrored using gmirror in a 6 way mirror (The goal of the boot and mirror redundancy is any drive can >die and I can still boot off any other drive like nothing happened. This partition contains the entire OS. >> - The zfs partitions are in RAIDZ-2 configuration and are redundant automatically. They contain the network accessible storage data. > >> My dilemma is this. I am upgrading to 5 TB Western Digital Black drives. I have replaced drive ada5 as a test. I used the -a 4k command while >partitioning to make sure sector alignment is correct. There are two major >> changes: > >> - ada5p3 is now 100 G >> - ada5p4 is now much larger due to the size of the drive > >> My understanding is that zfs will automatically change the total volume size once all drives are upgraded to the new 5 TB drives. Please correct >me if I'm wrong! The resilver went without a hitch. > > You may have to run "zpool online -e pool" once all the disk have been replaced, but yes it should be fairly easy to get ZFS to pick up the new space. > > The only other issue you may see is that if you built the original pool with 512b sectors (ashift 9) you may find "zpool status" start complaining that you are configured for 512b sectors when your disks are 4k (I haven't checked but considering the size I expect those 5TB disks are 4k). If that happens you either have to live with the warning or rebuild the pool. > >> My concern is with gmirror. Will gmirror grow to fit the new 100 G size automatically once the last drive is replaced? I got no errors using insert >with the 100 G partition into the mix with the other 5 30 G partitions. It synchronized fine. The volume shows as complete and all providers are >healthy. > > A quick test suggests you'll need to run "gmirror resize provider" once all the disks are replaced to get gmirror to update the size stored in the metadata - Good point. I didn't know about "gmirror resize". It was not in FreeBSD 8.4 - the last time I play with replacing by bigger disks. Thank you Miroslav Lachman From owner-freebsd-fs@freebsd.org Mon Jan 11 13:40:35 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF0CAA6B3DC for ; Mon, 11 Jan 2016 13:40:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A09A01739 for ; Mon, 11 Jan 2016 13:40:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BDeY1J063331 for ; Mon, 11 Jan 2016 13:40:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206109] zpool import of corrupt pool causes system to reboot Date: Mon, 11 Jan 2016 13:40:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 13:40:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206109 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 15:36:27 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20EB3A6ABBA for ; Mon, 11 Jan 2016 15:36:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 10EDE1F8E for ; Mon, 11 Jan 2016 15:36:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BFaQn6067410 for ; Mon, 11 Jan 2016 15:36:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 204358] zfs loader zfs_probe_args secsz is too small, causing memory corruption Date: Mon, 11 Jan 2016 15:36:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 15:36:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204358 --- Comment #2 from commit-hook@freebsd.org --- A commit references this bug: Author: allanjude Date: Mon Jan 11 15:35:30 UTC 2016 New revision: 293661 URL: https://svnweb.freebsd.org/changeset/base/293661 Log: DIOCGSECTORSIZE expects to write to a u_int, but struct zfs_probe_args member secsz was a uint16_t sys/boot/zfs/zfs.c has a probe args structure member, secsz, that is a uint16_t for media sector size; it is used as an argument for ioctl() at line 484. however, this ioctl writes 32 bits of data (u_int *) and therefore this ioctl will overwrite and corrupt 16 bits of memory. other use cases seem to use correct u_int type for secsz. PR: 204358 Submitted by: Toomas Soome Reviewed by: asomers, delphij, smh MFC after: 5 days Differential Revision: https://reviews.freebsd.org/D4811 Changes: head/sys/boot/zfs/zfs.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 16:08:58 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 704BCA6B46C for ; Mon, 11 Jan 2016 16:08:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5F7551289 for ; Mon, 11 Jan 2016 16:08:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BG8vNa074834 for ; Mon, 11 Jan 2016 16:08:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Mon, 11 Jan 2016 16:08:58 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 16:08:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #165366|0 |1 is obsolete| | --- Comment #10 from Pedro F. Giffuni --- Created attachment 165400 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D165400&action= =3Dedit support for EXT4 sparse files, cleaned up Starting from version 3, I cleaned up some style(9) issues. Also reused EXT2_MAX_BLOCK_SIZE instead of the hard coded 4096. I will commit this later unless there are objections. Any reason why it shouldn't work against 10.x ? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 16:52:09 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B597A6BD71 for ; Mon, 11 Jan 2016 16:52:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 05ED2157A for ; Mon, 11 Jan 2016 16:52:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BGq8GF098932 for ; Mon, 11 Jan 2016 16:52:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Mon, 11 Jan 2016 16:52:09 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 16:52:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 --- Comment #11 from Damjan Jovanovic --- (In reply to Pedro F. Giffuni from comment #10) Great :). I've tested it on PC-BSD 10.2 on a physical machine in addition to FreeBSD CURRENT on VirtualBox. It applies and works on both. I also managed to use = it to copy all 25 GB of files from my Linux partition with no problems, so it seems ready to be committed. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 18:58:23 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9FCBA6C3F6 for ; Mon, 11 Jan 2016 18:58:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 930D01627 for ; Mon, 11 Jan 2016 18:58:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BIwNwm030282 for ; Mon, 11 Jan 2016 18:58:23 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Mon, 11 Jan 2016 18:58:23 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 18:58:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |pfg@FreeBSD.org Assignee|freebsd-fs@FreeBSD.org |pfg@FreeBSD.org --- Comment #1 from Pedro F. Giffuni --- Take --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 19:15:47 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9DD4A6C83B for ; Mon, 11 Jan 2016 19:15:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DAE6A1464 for ; Mon, 11 Jan 2016 19:15:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BJFloi006113 for ; Mon, 11 Jan 2016 19:15:47 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Mon, 11 Jan 2016 19:15:47 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 19:15:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 --- Comment #12 from commit-hook@freebsd.org --- A commit references this bug: Author: pfg Date: Mon Jan 11 19:14:56 UTC 2016 New revision: 293680 URL: https://svnweb.freebsd.org/changeset/base/293680 Log: ext4: add support for reading sparse files Add support for sparse files in ext4. Also implement read-ahead, which greatly increases the performance when transferring files from ext4. Both features implemented by Damjan Jovanovic. PR: 205816 MFC after: 1 week Changes: head/sys/fs/ext2fs/ext2_bmap.c head/sys/fs/ext2fs/ext2_extents.c head/sys/fs/ext2fs/ext2_extents.h head/sys/fs/ext2fs/ext2_vnops.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 19:23:50 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7578A6CC9F for ; Mon, 11 Jan 2016 19:23:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 97EF01192 for ; Mon, 11 Jan 2016 19:23:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BJNoKd033090 for ; Mon, 11 Jan 2016 19:23:50 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Mon, 11 Jan 2016 19:23:50 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 19:23:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |mfc-stable9?, mfc-stable10? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 19:25:53 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F156A6CD32 for ; Mon, 11 Jan 2016 19:25:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 204C713C3 for ; Mon, 11 Jan 2016 19:25:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BJPqEp037754 for ; Mon, 11 Jan 2016 19:25:52 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Mon, 11 Jan 2016 19:25:53 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 19:25:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 --- Comment #2 from commit-hook@freebsd.org --- A commit references this bug: Author: pfg Date: Mon Jan 11 19:25:44 UTC 2016 New revision: 293683 URL: https://svnweb.freebsd.org/changeset/base/293683 Log: ext4: mount panic from freeing invalid pointers Initialize the struct with those fields to zeroes on allocation, preventing the panic. Patch by: Damjan Jovanovic. PR: 206056 MFC after: 3 days Changes: head/sys/fs/ext2fs/ext2_vfsops.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 19:30:05 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5FFEA6CE4B for ; Mon, 11 Jan 2016 19:30:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 96331198A for ; Mon, 11 Jan 2016 19:30:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BJU5VW048723 for ; Mon, 11 Jan 2016 19:30:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Mon, 11 Jan 2016 19:30:05 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: keywords flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 19:30:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |crash Flags| |mfc-stable9?, mfc-stable10? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 19:41:35 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09042A6B2CC for ; Mon, 11 Jan 2016 19:41:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E61AF1333 for ; Mon, 11 Jan 2016 19:41:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BJfYCV082874 for ; Mon, 11 Jan 2016 19:41:34 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206059] [ext2fs][patch] EXT4: cannot mount filesystems < 512 MiB in size: "ext2fs: no space for extra inode timestamps" Date: Mon, 11 Jan 2016 19:41:35 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 19:41:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206059 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |pfg@FreeBSD.org --- Comment #1 from Pedro F. Giffuni --- This is wrong: Look at struct ext2fs_dinode (ext2_dinode.h). If the inode size is 128, the extra timestamps really don't fit. Any attempt to read beyond the 128 would= be a buffer overflow. I recall the big block sizes were used for handling (non standard) extended attributes and the extra_isize only started getting used with ext4. I think this might be a bug in the GNU ext2fsprogs: technically you should disable the extra_isize with tune2fs before mounting such filesystems. --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 19:45:29 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B78BA6B391 for ; Mon, 11 Jan 2016 19:45:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 33D6D151F for ; Mon, 11 Jan 2016 19:45:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BJjSPZ095824 for ; Mon, 11 Jan 2016 19:45:29 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Mon, 11 Jan 2016 19:45:29 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10+ X-Bugzilla-Changed-Fields: flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 19:45:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|mfc-stable10? |mfc-stable10+ --- Comment #13 from Pedro F. Giffuni --- I am unsure if this should go into 9 as I can't really test. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 19:46:35 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8D663A6B426 for ; Mon, 11 Jan 2016 19:46:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7DD1315E9 for ; Mon, 11 Jan 2016 19:46:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BJkYem098266 for ; Mon, 11 Jan 2016 19:46:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Mon, 11 Jan 2016 19:46:35 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9+ mfc-stable10+ X-Bugzilla-Changed-Fields: bug_status flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 19:46:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |In Progress Flags|mfc-stable9?, mfc-stable10? |mfc-stable9+, mfc-stable10+ --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 20:14:51 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57CDBA6BD71 for ; Mon, 11 Jan 2016 20:14:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 483321D82 for ; Mon, 11 Jan 2016 20:14:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BKEof7014913 for ; Mon, 11 Jan 2016 20:14:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Mon, 11 Jan 2016 20:14:51 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 20:14:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|mfc-stable10+ |mfc-stable10? --- Comment #14 from Kubilay Kocak --- @Pedro The mfc-stableX flags are used to track when/after a change has been committed to the respective branch (+, meaning done), or in the case where = it wont, or shouldn't be (-, with comment). If you're not sure, leave the valu= e as ?=20 You can mouse-over the flag names for more detail and example usage/meaning --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 20:15:04 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F242A6BDA7 for ; Mon, 11 Jan 2016 20:15:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5FC4E1DF6 for ; Mon, 11 Jan 2016 20:15:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BKF3V5015477 for ; Mon, 11 Jan 2016 20:15:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Mon, 11 Jan 2016 20:15:04 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: feature, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: keywords Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 20:15:04 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |feature --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 20:17:08 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38300A6BE3F for ; Mon, 11 Jan 2016 20:17:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2897E1ECA for ; Mon, 11 Jan 2016 20:17:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BKH8HX020532 for ; Mon, 11 Jan 2016 20:17:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Mon, 11 Jan 2016 20:17:08 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 20:17:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|mfc-stable9+, mfc-stable10+ |mfc-stable9?, mfc-stable10? --- Comment #3 from Kubilay Kocak --- Leaving mfc-stableX flags ? until committed in those branches as per bug 20= 5816 Commetn 14 --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 20:17:28 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8EB5A6BE79 for ; Mon, 11 Jan 2016 20:17:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 997A11F4A for ; Mon, 11 Jan 2016 20:17:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0BKHS44021348 for ; Mon, 11 Jan 2016 20:17:28 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Mon, 11 Jan 2016 20:17:28 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 20:17:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 --- Comment #4 from Kubilay Kocak --- I mean bug 205816 Comment 14 --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 11 20:42:54 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 23284A6C8B3 for ; Mon, 11 Jan 2016 20:42:54 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E8C201435; Mon, 11 Jan 2016 20:42:53 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id u0BKgka2004072; Mon, 11 Jan 2016 14:42:46 -0600 (CST) Date: Mon, 11 Jan 2016 14:42:46 -0600 (CST) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: kpneal@pobox.com cc: Lev Serebryakov , FreeBSD Filesystems Subject: Re: Restore only several files from ZFS snapshot without creating copy of them? In-Reply-To: <20160111203257.GB88498@neutralgood.org> Message-ID: References: <133976260.20160110023807@serebryakov.spb.ru> <20160111203257.GB88498@neutralgood.org> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Mon, 11 Jan 2016 14:42:46 -0600 (CST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 20:42:54 -0000 On Mon, 11 Jan 2016, kpneal@pobox.com wrote: > > I believe you are stuck between the proverbial rock and a hard place. You > have two options: > > 1) Have two copies of the files by making a copy from the snapshot like > you said OR The rock is somewhat softer than that. All that is needed to remove most of the duplication is a file copy program which operates at the filesystem blocksize (e.g. 128k) and only copies blocks for existing files if the data in that block has changed. Absent writing a dedicated copy program, rsync with options '-a --inplace --no-whole-file' comes close to copying the file without duplication since it compares blocks and only writes changed blocks (but not necessarily aligned to zfs blocks). Use 'zfs diff' to quickly see which files have changed. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@freebsd.org Mon Jan 11 21:20:59 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD539A6B885 for ; Mon, 11 Jan 2016 21:20:59 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: from mail-qk0-x22f.google.com (mail-qk0-x22f.google.com [IPv6:2607:f8b0:400d:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 714D21A8E for ; Mon, 11 Jan 2016 21:20:59 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: by mail-qk0-x22f.google.com with SMTP id n135so231409810qka.2 for ; Mon, 11 Jan 2016 13:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kraus-haus-org.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=l412M7nN5/41pOJc6Aa56yA/nIh/ZD2SBTxrfxuqOKM=; b=uMdc0pSaz17mEn8/xWc9y8gcY9+qWp3BXfyl54jKa/HfAQqlvnYQBnnm6g/M0YhLe+ M9GxV1DRiZh9e8we9rh8e+HkHedl+CQBqzeXbrknnhYBRzf1SHkvpj16JqImx63gGm8Z O1xBB8NaBs3EVyPq6I40/XnfnuoIl8dWW55GekA4CdKP5NkMV6B816vyvuWeOGkilJhM 2uibUtMPLP6/ewuIwtHfsHYaEJ92Q0KmmV7+TE6zU8wtQhJ2WLkvsC6s5P+YY7beDOtH A2PneMvSGdzag0HcdgJ2gr5vPiXkRcAI2Fjb8G4SngRmhx/w6CCNpu/Z8auKNOPphdhx Jj1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=l412M7nN5/41pOJc6Aa56yA/nIh/ZD2SBTxrfxuqOKM=; b=ldWyFrsqrYeOFWi7rDONRSReCPEPViGoB2eEJqsZ1tVjCGwF6qB9163db3SGoP5lgp HLDiFlB55Xfwc+9vGSLX7q85Jij09T5Lc8kB5r9kGvnzBzR0fDrCPz65qHjhD66GU/UJ ZG0SlrPtKqhkgmPSnmyZOhdT+VdIMPf9raBHk9vEFjIbn6kOd7kKcriUSUgtotqgFofg LtmyB7hqPGTm+fIX0N3gSRPU8n1odtVdA8he0iMZZFL0noSGVsVdCUF3z/mAkv0flpaJ RsDGZCMDWZsO/nRiEtDaKGhZOV5uc9F0KNXz6o8wiGIql8swFaXrj8C9ApUyG489nkHS Rzwg== X-Gm-Message-State: ALoCoQlZd8QiEwJk8lijpGKBtJoMpFUvT2fTVk9zcP2DtMbN/Li6duK1I7RUz3d0pCx7zMla2W1MaQnJckoaJ/u+L7JHLW/uRw== X-Received: by 10.55.80.86 with SMTP id e83mr164642787qkb.91.1452547258332; Mon, 11 Jan 2016 13:20:58 -0800 (PST) Received: from mbp-1.thecreativeadvantage.com (mail.thecreativeadvantage.com. [96.236.20.34]) by smtp.gmail.com with ESMTPSA id j67sm47258239qgj.35.2016.01.11.13.20.57 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 11 Jan 2016 13:20:57 -0800 (PST) Subject: Re: Restore only several files from ZFS snapshot without creating copy of them? Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Content-Type: text/plain; charset=us-ascii From: Paul Kraus X-Priority: 3 (Normal) In-Reply-To: <133976260.20160110023807@serebryakov.spb.ru> Date: Mon, 11 Jan 2016 16:20:51 -0500 Cc: FreeBSD Filesystems Content-Transfer-Encoding: quoted-printable Message-Id: References: <133976260.20160110023807@serebryakov.spb.ru> To: lev@FreeBSD.org X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 21:20:59 -0000 On Jan 9, 2016, at 18:38, Lev Serebryakov wrote: > Hello FreeBSD, >=20 > I have a ZFS filesystem, which have daily snapshots. User removed = several > multi-gigabyte files by accident and need to get them back. These = files are > present in old daily snapshots, allright. But as far as I understnad, = if I > do >=20 > cp /fs/.zfs/snapshots/old-snapshot/file /fs/file >=20 > I got TWO copies of this file (one in live filesystem and future = snapshots > and other one in old snapshots). As files in question are = multi-gigabyte > (about 100G alltogether) I don't want such duplication (dedup is = turned off, > as it is very memory-consuming). >=20 > Is it possible to restore these files without such data duplication? Could you leverage a CLONE of the snapshot until such time as other = snapshots of these large files have expired (assuming you are expiring = and destroying old snapshots after a certain amount of time) ? I am = trying to think out of the box here and have to completely thought all = this through.=20 -- Paul Kraus paul@kraus-haus.org From owner-freebsd-fs@freebsd.org Mon Jan 11 21:39:09 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D74D9A6C036 for ; Mon, 11 Jan 2016 21:39:09 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: from mail-qg0-x236.google.com (mail-qg0-x236.google.com [IPv6:2607:f8b0:400d:c04::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A45BF1616; Mon, 11 Jan 2016 21:39:09 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: by mail-qg0-x236.google.com with SMTP id e32so324981102qgf.3; Mon, 11 Jan 2016 13:39:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=hPvqO/SM7UVcbvCtak38KvjEOLgmk+SmUUwypGvk8wY=; b=F1FP7eYTlEMgi3OeHgFCzuZEk8ufdbuDcSZwHBCqyhmYREC6keQN9gKBwdD5JUR119 CuOUJdASeHXQGKdLzQwrxX47hKdbw4lDg0rrPYqoU6IbJY95ues/rOlUCtACus3kfELe Fcc36y0v9D8CQ87bL02XIRl1ckU19KU2g0vbBs9Oar/RL7l1w0xkdDhYCF3mVtoAiJgb oKMCJDhh1hmB4+Pzj8XOsGqzZzaln3O21rW8PeAWdy7f3jOT78Iwjtu57SMfW5cJvQU/ tgjvWkZDUSb8VKfM9R1JS18tQvDNzu6E8bFCB63+GSocNpsJJkbxkCUW5rQW855Q2GZF /INg== MIME-Version: 1.0 X-Received: by 10.140.232.78 with SMTP id d75mr158713096qhc.27.1452548348734; Mon, 11 Jan 2016 13:39:08 -0800 (PST) Received: by 10.55.214.4 with HTTP; Mon, 11 Jan 2016 13:39:08 -0800 (PST) In-Reply-To: References: <133976260.20160110023807@serebryakov.spb.ru> Date: Mon, 11 Jan 2016 15:39:08 -0600 Message-ID: Subject: Re: Restore only several files from ZFS snapshot without creating copy of them? From: "Eric A. Borisch" To: Paul Kraus Cc: lev@freebsd.org, FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 21:39:09 -0000 On Mon, Jan 11, 2016 at 3:20 PM, Paul Kraus wrote: > On Jan 9, 2016, at 18:38, Lev Serebryakov wrote: > >> Hello FreeBSD, >> >> I have a ZFS filesystem, which have daily snapshots. User removed seve= ral >> multi-gigabyte files by accident and need to get them back. These files = are >> present in old daily snapshots, allright. But as far as I understnad, if= I >> do >> >> cp /fs/.zfs/snapshots/old-snapshot/file /fs/file >> >> I got TWO copies of this file (one in live filesystem and future snaps= hots >> and other one in old snapshots). As files in question are multi-gigabyte >> (about 100G alltogether) I don't want such duplication (dedup is turned = off, >> as it is very memory-consuming). >> >> Is it possible to restore these files without such data duplication? > > Could you leverage a CLONE of the snapshot until such time as other snaps= hots of these large files have expired (assuming you are expiring and destr= oying old snapshots after a certain amount of time) ? I am trying to think = out of the box here and have to completely thought all this through. I think you'll find a combination of Paul's and Bob's answers will be your best bet; clone, then rsync (or some other update tool) into the clone from the 'current' (with appropriate --exclude rules to not delete the files in question, likely any other deletions will be desired) and then rename/promote/delete when ready. - Eric From owner-freebsd-fs@freebsd.org Mon Jan 11 22:40:24 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B2DEA6B81F for ; Mon, 11 Jan 2016 22:40:24 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: from mail-qg0-x232.google.com (mail-qg0-x232.google.com [IPv6:2607:f8b0:400d:c04::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F5B61EF2; Mon, 11 Jan 2016 22:40:24 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: by mail-qg0-x232.google.com with SMTP id 6so336823987qgy.1; Mon, 11 Jan 2016 14:40:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vRMms0JS0Roc2J6YV+pkyDMo1Yq1yfaLhYQ5eZc5DV0=; b=y3wWh3RKMm5dYqfkbx85UoLVXLPQFWXiTuwjSiXfub3aTTqRcvezFpoCLMc7aTawUG uH23jTlcJpFPFQuYfjOlSnozJ8XeiU23CCi1+HHfrihBLlQAfCm855VotFmEVvGl/kVm XKRAwn47mMg/KDBVIbZT8rRxpEc9u6uI4ksqtRBRYD8JelqIvpctR7pHTJfRl3Ws3nAh kGV2k8pOGndkxExAhI0tdMTOOpJwtPzze7IkqFbpLngSJ1Y7ccrlHwUh/t8wLXjQJNra FJPuOJmLoqSIvhNst1uEQ9DeD+s7w5l8Y1ScKzahU4vhP0/nhDTQ4Xxc6f1ylKCM3585 2ffA== MIME-Version: 1.0 X-Received: by 10.140.98.53 with SMTP id n50mr171772279qge.9.1452552023451; Mon, 11 Jan 2016 14:40:23 -0800 (PST) Received: by 10.55.214.4 with HTTP; Mon, 11 Jan 2016 14:40:23 -0800 (PST) In-Reply-To: <20160111222436.GE88498@neutralgood.org> References: <133976260.20160110023807@serebryakov.spb.ru> <20160111222436.GE88498@neutralgood.org> Date: Mon, 11 Jan 2016 16:40:23 -0600 Message-ID: Subject: Re: Restore only several files from ZFS snapshot without creating copy of them? From: "Eric A. Borisch" To: kpneal@pobox.com Cc: Paul Kraus , FreeBSD Filesystems , lev@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 22:40:24 -0000 On Mon, Jan 11, 2016 at 4:24 PM, wrote: > But can a clone be promoted to the "head" of a dataset? I thought a clone > was always backed by a snapshot. The goal as I understood it was to get > the files back into the normal dataset. >From man zfs: zfs promote clone-filesystem Promotes a clone file system to no longer be dependent on its "ori- gin" snapshot. This makes it possible to destroy the file system that the clone was created from. The clone parent-child dependency rela- tionship is reversed, so that the origin file system becomes a clone of the specified file system. What you will lose in the process is the snapshot history between the 'pre whoops' snapshot and 'now.' (After recreating the state of 'now' in the clone of 'pre whoops' (less the accidental removals) via rsync, for example.) - Eric From owner-freebsd-fs@freebsd.org Mon Jan 11 22:40:38 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D15AA6B86D for ; Mon, 11 Jan 2016 22:40:38 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DB60E1F70; Mon, 11 Jan 2016 22:40:37 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aIl8d-000BXC-DO; Mon, 11 Jan 2016 22:40:35 +0000 Date: Mon, 11 Jan 2016 22:40:35 +0000 From: Gary Palmer To: kpneal@pobox.com Cc: "Eric A. Borisch" , FreeBSD Filesystems , lev@freebsd.org, Paul Kraus Subject: Re: Restore only several files from ZFS snapshot without creating copy of them? Message-ID: <20160111224035.GA1019@in-addr.com> References: <133976260.20160110023807@serebryakov.spb.ru> <20160111222436.GE88498@neutralgood.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160111222436.GE88498@neutralgood.org> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 22:40:38 -0000 On Mon, Jan 11, 2016 at 05:24:36PM -0500, kpneal@pobox.com wrote: > On Mon, Jan 11, 2016 at 03:39:08PM -0600, Eric A. Borisch wrote: > > On Mon, Jan 11, 2016 at 3:20 PM, Paul Kraus wrote: > > > On Jan 9, 2016, at 18:38, Lev Serebryakov wrote: > > > > > >> Hello FreeBSD, > > >> > > >> I have a ZFS filesystem, which have daily snapshots. User removed several > > >> multi-gigabyte files by accident and need to get them back. These files are > > >> present in old daily snapshots, allright. But as far as I understnad, if I > > >> do > > >> > > >> cp /fs/.zfs/snapshots/old-snapshot/file /fs/file > > >> > > >> I got TWO copies of this file (one in live filesystem and future snapshots > > >> and other one in old snapshots). As files in question are multi-gigabyte > > >> (about 100G alltogether) I don't want such duplication (dedup is turned off, > > >> as it is very memory-consuming). > > >> > > >> Is it possible to restore these files without such data duplication? > > > > > > Could you leverage a CLONE of the snapshot until such time as other snapshots of these large files have expired (assuming you are expiring and destroying old snapshots after a certain amount of time) ? I am trying to think out of the box here and have to completely thought all this through. > > > > I think you'll find a combination of Paul's and Bob's answers will be > > your best bet; clone, then rsync (or some other update tool) into the > > clone from the 'current' (with appropriate --exclude rules to not > > delete the files in question, likely any other deletions will be > > desired) and then rename/promote/delete when ready. > > But can a clone be promoted to the "head" of a dataset? I thought a clone > was always backed by a snapshot. The goal as I understood it was to get > the files back into the normal dataset. zfs promote Regards Gary > Now, having a clone mounted with symlinks from the normal dataset to the > mounted clone may work. It depends on whether or not the files get changed. > (The point of the clone is to avoid having the snapshot deleted out from > under you.) > > Another idea would be to copy the files from the snapshot into the normal > dataset and then to delete the old snapshots. This will leave you without > the safety net of the old snapshots, but it will also leave you with one > copy of the files in the normal location. It's not my place to say if that > is acceptable or not. > > -- > Kevin P. Neal http://www.pobox.com/~kpn/ > > "It sounded pretty good, but it's hard to tell how it will work out > in practice." -- Dennis Ritchie, ~1977, "Summary of a DEC 32-bit machine" > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Mon Jan 11 23:44:19 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90C7CA6C289 for ; Mon, 11 Jan 2016 23:44:19 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 606881882 for ; Mon, 11 Jan 2016 23:44:18 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id u0BNiHdt004582; Mon, 11 Jan 2016 17:44:17 -0600 (CST) Date: Mon, 11 Jan 2016 17:44:17 -0600 (CST) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: kpneal@pobox.com cc: FreeBSD Filesystems Subject: Re: Restore only several files from ZFS snapshot without creating copy of them? In-Reply-To: <20160111222911.GF88498@neutralgood.org> Message-ID: References: <133976260.20160110023807@serebryakov.spb.ru> <20160111203257.GB88498@neutralgood.org> <20160111222911.GF88498@neutralgood.org> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Mon, 11 Jan 2016 17:44:17 -0600 (CST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2016 23:44:19 -0000 On Mon, 11 Jan 2016, kpneal@pobox.com wrote: >> >> The rock is somewhat softer than that. All that is needed to remove >> most of the duplication is a file copy program which operates at the >> filesystem blocksize (e.g. 128k) and only copies blocks for existing >> files if the data in that block has changed. >> >> Absent writing a dedicated copy program, rsync with options '-a >> --inplace --no-whole-file' comes close to copying the file without >> duplication since it compares blocks and only writes changed blocks >> (but not necessarily aligned to zfs blocks). > > I don't think that will work to restore files from snapshots as snapshots > appear as different filesystems to applications if I'm not mistaken. That's > also why using "ln" to create a hard link won't work. Unless I'm mistaken. Rsync just copies files from one location to another, and/or tries make the copy as similar to the original with minimal work. It is easy to rsync from a snapshot (from /filesystem/.zfs/snapshot/snapshotname). If there have been large changes to the file and there are intermediate snapshots, then even using rsync with '--inplace --no-whole-file' will still result in losing the space attributed to those intermediate snapshots. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@freebsd.org Wed Jan 13 15:16:30 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B588A815D8 for ; Wed, 13 Jan 2016 15:16:30 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 1406210E4 for ; Wed, 13 Jan 2016 15:16:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:OAicoBKtmGRrJhBRZdmcpTZWNBhigK39O0sv0rFitYgULPzxwZ3uMQTl6Ol3ixeRBMOAu6wC27Wd7fGocFdDyKjCmUhKSIZLWR4BhJdetC0bK+nBN3fGKuX3ZTcxBsVIWQwt1Xi6NU9IBJS2PAWK8TWM5DIfUi/yKRBybrysXNWC0YLniKvsqtX6WEZhunmUWftKNhK4rAHc5IE9oLBJDeIP8CbPuWZCYO9MxGlldhq5lhf44dqsrtY4q3wD86Fpy8kVcqL8ZLgxS6BZCnwMPmQy+dbsq1GXTgyU+nofWGgSuhVNCgnBqhr9W8GinDH9s79H2SKZdej/RrMwVDHqu71uQRTrjCoCHyM+/3zajtRwyqlS9kHy7ydjypLZNdnGfMF1ebnQKIsX X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CoBAA7aZZW/61jaINehH+IVLUPhg8CgXERAQEBAQEBAQGBCYItgggBAQQjBFIQAgEIDgoCAg0ZAgJXAgSIQbAUkDgBAQEBAQUBAQEBAQEdgQGFVYR/h3SBSQWOOIhdjziERIheRIodg3ECOCyCERyBeyCFSIEIAQEB X-IronPort-AV: E=Sophos;i="5.22,289,1449550800"; d="scan'208";a="262578310" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 13 Jan 2016 10:16:22 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 7322815F565; Wed, 13 Jan 2016 10:16:22 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id DmMfbDD5IdpI; Wed, 13 Jan 2016 10:16:21 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 7F84415F56D; Wed, 13 Jan 2016 10:16:21 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id eYqweQl5gShV; Wed, 13 Jan 2016 10:16:21 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 58AE615F565; Wed, 13 Jan 2016 10:16:21 -0500 (EST) Date: Wed, 13 Jan 2016 10:16:21 -0500 (EST) From: Rick Macklem To: Konstantin Belousov Cc: FreeBSD Filesystems Message-ID: <1773157955.158922767.1452698181137.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20160110154518.GU3625@kib.kiev.ua> References: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> <20160110154518.GU3625@kib.kiev.ua> Subject: Re: panic ffs_truncate3 (maybe fuse being evil) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: panic ffs_truncate3 (maybe fuse being evil) Thread-Index: 5I8dlwX1ds6laP1djmSqjnkEIr28EQ== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2016 15:16:30 -0000 Kostik wrote: > On Sun, Jan 10, 2016 at 10:01:57AM -0500, Rick Macklem wrote: > > Hi, > > > > When fooling around with GlusterFS, I can get this panic intermittently. > > (I had a couple yesterday.) This happens on a Dec. 5, 2015 head kernel. > > > > panic: ffs_truncate3 > > - backtrace without the numbers (I just scribbled it off the screen) > > ffs_truncate() > > ufs_inactive() > > VOP_INACTIVE_APV() > > vinactive() > > vputx() > > kern_unlinkat() > > > > So, at a glance, it seems that either > > b_dirty.bv_cnt > > or b_clean.bv_cnt > > is non-zero. (There is another case for the panic, but I thought it > > was less likely?) > > > > So, I'm wondering if this might be another side effect of r291460, > > since after that a new vnode isn't completely zero'd out? > > > > However, shouldn't bo_dirty.bv_cnt and bo_clean.bv_cnt be zero when > > a vnode is recycled? > > Does this make sense or do some fields of v_bufobj need to be zero'd > > out by getnewvnode()? > Look at the _vdrop(). When a vnode is freed to zone, it is asserted > that bufobj queues are empty. I very much doubt that it is possible > to leak either buffers or counters by reuse. > > > > > GlusterFS is using fuse and I suspect that fuse isn't cleaning out > > the buffers under some circumstance (I already noticed that there > > isn't any code in its fuse_vnop_reclaim() and I vaguely recall that > > there are conditions where VOP_INACTIVE() gets skipped, so that > > VOP_RECLAIM() > > has to check for anything that would have been done by VOP_INACTIVE() > > and do it, if it isn't already done.) > But even if fuse leaves the buffers around, is it UFS which panics for > you ? I would rather worry about dandling pointers and use after free in > fuse, which is a known issue with it anyway. I.e. it could be that fuse > operates on reclaimed and reused vnode as its own. > > > > > Anyhow, if others have thoughts on this (or other hunches w.r.t. what > > could cause this panic(), please let me know. > > The ffs_truncate3 was deterministically triggered by a bug in ffs_balloc(). > The routine allocated buffers for indirect blocks, but if the blocks cannot > be allocated, the buffers where left on queue. See r174973, this was fixed > very long time ago. > Well, although I have r174973 in the kernel that crashes, it looks like this bug might have been around for a while. Here's what I've figured out sofar. 1 - The crashes only occur if soft updates are disabled. This isn't surprising if you look at ffs_truncate(), since the test for the panic isn't done when soft updates are enabled. Here's the snippet from ffs_truncate(), in case you are interested: if (DOINGSOFTDEP(vp)) { 335 if (softdeptrunc == 0 && journaltrunc == 0) { 336 /* 337 * If a file is only partially truncated, then 338 * we have to clean up the data structures 339 * describing the allocation past the truncation 340 * point. Finding and deallocating those structures 341 * is a lot of work. Since partial truncation occurs 342 * rarely, we solve the problem by syncing the file 343 * so that it will have no data structures left. 344 */ 345 if ((error = ffs_syncvnode(vp, MNT_WAIT, 0)) != 0) 346 return (error); 347 } else { 348 flags = IO_NORMAL | (needextclean ? IO_EXT: 0); 349 if (journaltrunc) 350 softdep_journal_freeblocks(ip, cred, length, 351 flags); 352 else 353 softdep_setup_freeblocks(ip, length, flags); 354 ASSERT_VOP_LOCKED(vp, "ffs_truncate1"); 355 if (journaltrunc == 0) { 356 ip->i_flag |= IN_CHANGE | IN_UPDATE; 357 error = ffs_update(vp, 0); 358 } 359 return (error); 360 } 361 } You can see that it always returns once in this code block. The only way the code can get past this block if soft updates are enabled is a "goto extclean;", which takes you past the "panic()". By adding a few printf()s, I have determined: - The bo_clean.bv_cnt == 1 when the panic occurs and the b_lblkno of the buffer is -ve. If you look at vtruncbuf(): trunclbn = (length + blksize - 1) / blksize; 1726 1727 ASSERT_VOP_LOCKED(vp, "vtruncbuf"); 1728 restart: 1729 bo = &vp->v_bufobj; 1730 BO_LOCK(bo); 1731 anyfreed = 1; 1732 for (;anyfreed;) { 1733 anyfreed = 0; 1734 TAILQ_FOREACH_SAFE(bp, &bo->bo_clean.bv_hd, b_bobufs, nbp) { 1735 if (bp->b_lblkno < trunclbn) 1736 continue; When length == 0 --> trunclbn is 0, but the test at line#1735 will skip over the b_lblkno because it is negative. That is as far as I've gotten. A couple of things I need help from others on: - Is vtruncbuf() skipping over the cases where b_lblkno < 0 a feature or a bug? - If it is a feature, then what needs to be done in the code after the vtruncbuf() call in ffs_truncate() to ensure the buffer is gone by the time the panic check is done? --> I do see a bunch of code after the vtruncbuf() call related to indirect blocks (which I think use the -ve b_lblkno?), but I'll admit I don't understand it well enough to know if it expects vtruncbuf() to leave the -ve block on the bo_hd list? Obviously fixing vtruncbuf() to get rid of these -ve b_lblkno entries would be easy, but I don't know if that is a feature or a bug? I did look at the commit logs and vtruncbuf() has been like this for at least 10years. (I can only guess very few run UFS without soft updates or others would see these panic()s.) I am now running with soft updates enabled to avoid the crashes, but I can easily test any patch if others can a patch to try. Thanks for your help with this, rick From owner-freebsd-fs@freebsd.org Wed Jan 13 15:40:28 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2AFDDA81D64 for ; Wed, 13 Jan 2016 15:40:28 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id D3BF71E55 for ; Wed, 13 Jan 2016 15:40:27 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:ECJCehLQZ/wpRAgkQtmcpTZWNBhigK39O0sv0rFitYgULPzxwZ3uMQTl6Ol3ixeRBMOAu6wC27Wd7fGocFdDyKjCmUhKSIZLWR4BhJdetC0bK+nBN3fGKuX3ZTcxBsVIWQwt1Xi6NU9IBJS2PAWK8TWM5DIfUi/yKRBybrysXNWC0YLniKvsqtX6WEZhunmUWftKNhK4rAHc5IE9oLBJDeIP8CbPuWZCYO9MxGlldhq5lhf44dqsrtY4q3wD86Fpy8kVcqL8ZLgxS6BZCnwMPmQy+dbsq1GXTgyU+nofWGgSuhVNCgnBqhr9W8GinDH9s79H2SKZdej/RrMwVDHqu71uQRTrjCoCHyM+/3zajtRwyqlS9kHy7ydjypLZNdnGfMF1ebnQKIsX X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CuBACdbpZW/61jaINehAxtBohUsy6BZBgKhSNKAoFwEgEBAQEBAQEBgQmCLYIHAQEBAwEBAQEgBCcgCwULAgEIDgoCAg0ZAgInAQkmAgQIBwQBHASIBQgOsAWQOgEBAQEBAQQBAQEBAQEZBIEBhVWEf4Q3AQEcgx+BSQWOOIhdhUOFK4RKhESIXkSKH4NxAikJMoIRHIF7IDQHhFM6gQgBAQE X-IronPort-AV: E=Sophos;i="5.22,289,1449550800"; d="scan'208";a="261005595" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 13 Jan 2016 10:40:18 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 637DB15F565; Wed, 13 Jan 2016 10:40:18 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 2iDgMN0Tgr4d; Wed, 13 Jan 2016 10:40:17 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 5BDA015F56D; Wed, 13 Jan 2016 10:40:17 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id GwJYeX06o9dB; Wed, 13 Jan 2016 10:40:17 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 3EF4115F565; Wed, 13 Jan 2016 10:40:17 -0500 (EST) Date: Wed, 13 Jan 2016 10:40:17 -0500 (EST) From: Rick Macklem To: Konstantin Belousov Cc: FreeBSD Filesystems Message-ID: <1351730674.159022044.1452699617235.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <1773157955.158922767.1452698181137.JavaMail.zimbra@uoguelph.ca> References: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> <20160110154518.GU3625@kib.kiev.ua> <1773157955.158922767.1452698181137.JavaMail.zimbra@uoguelph.ca> Subject: Re: panic ffs_truncate3 (maybe fuse being evil) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: panic ffs_truncate3 (maybe fuse being evil) Thread-Index: 5I8dlwX1ds6laP1djmSqjnkEIr28EdIJ2VW4 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2016 15:40:28 -0000 I wrote: > Kostik wrote: > > On Sun, Jan 10, 2016 at 10:01:57AM -0500, Rick Macklem wrote: > > > Hi, > > > > > > When fooling around with GlusterFS, I can get this panic intermittently. > > > (I had a couple yesterday.) This happens on a Dec. 5, 2015 head kernel. > > > > > > panic: ffs_truncate3 > > > - backtrace without the numbers (I just scribbled it off the screen) > > > ffs_truncate() > > > ufs_inactive() > > > VOP_INACTIVE_APV() > > > vinactive() > > > vputx() > > > kern_unlinkat() > > > > > > So, at a glance, it seems that either > > > b_dirty.bv_cnt > > > or b_clean.bv_cnt > > > is non-zero. (There is another case for the panic, but I thought it > > > was less likely?) > > > > > > So, I'm wondering if this might be another side effect of r291460, > > > since after that a new vnode isn't completely zero'd out? > > > > > > However, shouldn't bo_dirty.bv_cnt and bo_clean.bv_cnt be zero when > > > a vnode is recycled? > > > Does this make sense or do some fields of v_bufobj need to be zero'd > > > out by getnewvnode()? > > Look at the _vdrop(). When a vnode is freed to zone, it is asserted > > that bufobj queues are empty. I very much doubt that it is possible > > to leak either buffers or counters by reuse. > > > > > > > > GlusterFS is using fuse and I suspect that fuse isn't cleaning out > > > the buffers under some circumstance (I already noticed that there > > > isn't any code in its fuse_vnop_reclaim() and I vaguely recall that > > > there are conditions where VOP_INACTIVE() gets skipped, so that > > > VOP_RECLAIM() > > > has to check for anything that would have been done by VOP_INACTIVE() > > > and do it, if it isn't already done.) > > But even if fuse leaves the buffers around, is it UFS which panics for > > you ? I would rather worry about dandling pointers and use after free in > > fuse, which is a known issue with it anyway. I.e. it could be that fuse > > operates on reclaimed and reused vnode as its own. > > > > > > > > Anyhow, if others have thoughts on this (or other hunches w.r.t. what > > > could cause this panic(), please let me know. > > > > The ffs_truncate3 was deterministically triggered by a bug in ffs_balloc(). > > The routine allocated buffers for indirect blocks, but if the blocks cannot > > be allocated, the buffers where left on queue. See r174973, this was fixed > > very long time ago. > > > Well, although I have r174973 in the kernel that crashes, it looks like this > bug might have been around for a while. > Here's what I've figured out sofar. > 1 - The crashes only occur if soft updates are disabled. This isn't > surprising > if you look at ffs_truncate(), since the test for the panic isn't done > when soft updates are enabled. > Here's the snippet from ffs_truncate(), in case you are interested: > if (DOINGSOFTDEP(vp)) { > 335 if (softdeptrunc == 0 && journaltrunc == 0) { > 336 /* > 337 * If a file is only partially truncated, then > 338 * we have to clean up the data structures > 339 * describing the allocation past the truncation > 340 * point. Finding and deallocating those > structures > 341 * is a lot of work. Since partial truncation > occurs > 342 * rarely, we solve the problem by syncing the > file > 343 * so that it will have no data structures left. > 344 */ > 345 if ((error = ffs_syncvnode(vp, MNT_WAIT, 0)) != > 0) > 346 return (error); > 347 } else { > 348 flags = IO_NORMAL | (needextclean ? IO_EXT: 0); > 349 if (journaltrunc) > 350 softdep_journal_freeblocks(ip, cred, > length, > 351 flags); > 352 else > 353 softdep_setup_freeblocks(ip, length, > flags); > 354 ASSERT_VOP_LOCKED(vp, "ffs_truncate1"); > 355 if (journaltrunc == 0) { > 356 ip->i_flag |= IN_CHANGE | IN_UPDATE; > 357 error = ffs_update(vp, 0); > 358 } > 359 return (error); > 360 } > 361 } > You can see that it always returns once in this code block. The only way the > code can get > past this block if soft updates are enabled is a "goto extclean;", which > takes you past > the "panic()". > > By adding a few printf()s, I have determined: > - The bo_clean.bv_cnt == 1 when the panic occurs and the b_lblkno of the > buffer is -ve. > > If you look at vtruncbuf(): > trunclbn = (length + blksize - 1) / blksize; > 1726 > 1727 ASSERT_VOP_LOCKED(vp, "vtruncbuf"); > 1728 restart: > 1729 bo = &vp->v_bufobj; > 1730 BO_LOCK(bo); > 1731 anyfreed = 1; > 1732 for (;anyfreed;) { > 1733 anyfreed = 0; > 1734 TAILQ_FOREACH_SAFE(bp, &bo->bo_clean.bv_hd, b_bobufs, > nbp) { > 1735 if (bp->b_lblkno < trunclbn) > 1736 continue; > When length == 0 --> trunclbn is 0, but the test at line#1735 will skip over > the b_lblkno > because it is negative. > > That is as far as I've gotten. A couple of things I need help from others on: > - Is vtruncbuf() skipping over the cases where b_lblkno < 0 a feature or a > bug? > - If it is a feature, then what needs to be done in the code after the > vtruncbuf() > call in ffs_truncate() to ensure the buffer is gone by the time the panic > check is > done? > --> I do see a bunch of code after the vtruncbuf() call related to indirect > blocks > (which I think use the -ve b_lblkno?), but I'll admit I don't understand > it well > enough to know if it expects vtruncbuf() to leave the -ve block on the > bo_hd list? > > Obviously fixing vtruncbuf() to get rid of these -ve b_lblkno entries would > be easy, > but I don't know if that is a feature or a bug? > > I did look at the commit logs and vtruncbuf() has been like this for at least > 10years. > (I can only guess very few run UFS without soft updates or others would see > these panic()s.) > > I am now running with soft updates enabled to avoid the crashes, but I can > easily test any > patch if others can a patch to try. > Oh, and one more thing. Maybe having the buffer for an indirect block hanging off the vnode at the end of ffs_truncate() to 0 length is ok. After all, this is happening in VOP_INACTIVE() and the vnode isn't being recycled yet? (ie. The panic() test is not needed?) rick > Thanks for your help with this, rick > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Thu Jan 14 01:50:42 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A11CEA823E0 for ; Thu, 14 Jan 2016 01:50:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 913A81EAF for ; Thu, 14 Jan 2016 01:50:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0E1ofqr076220 for ; Thu, 14 Jan 2016 01:50:42 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Thu, 14 Jan 2016 01:50:41 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2016 01:50:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 --- Comment #5 from commit-hook@freebsd.org --- A commit references this bug: Author: pfg Date: Thu Jan 14 01:50:06 UTC 2016 New revision: 293866 URL: https://svnweb.freebsd.org/changeset/base/293866 Log: MFC r293683: ext4: mount panic from freeing invalid pointers Initialize the struct with those fields to zeroes on allocation, preventing the panic. Patch by: Damjan Jovanovic. PR: 206056 Changes: _U stable/10/ stable/10/sys/fs/ext2fs/ext2_vfsops.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 14 01:51:44 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2752A82566 for ; Thu, 14 Jan 2016 01:51:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C2CF0123F for ; Thu, 14 Jan 2016 01:51:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0E1pieH006673 for ; Thu, 14 Jan 2016 01:51:44 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Thu, 14 Jan 2016 01:51:44 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2016 01:51:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 --- Comment #6 from commit-hook@freebsd.org --- A commit references this bug: Author: pfg Date: Thu Jan 14 01:51:18 UTC 2016 New revision: 293867 URL: https://svnweb.freebsd.org/changeset/base/293867 Log: MFC r293683: ext4: mount panic from freeing invalid pointers Initialize the struct with those fields to zeroes on allocation, preventing the panic. Patch by: Damjan Jovanovic. PR: 206056 Changes: _U stable/9/sys/ _U stable/9/sys/fs/ stable/9/sys/fs/ext2fs/ext2_vfsops.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 14 01:54:42 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A8D0A82665 for ; Thu, 14 Jan 2016 01:54:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 598051378 for ; Thu, 14 Jan 2016 01:54:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0E1sfIO076880 for ; Thu, 14 Jan 2016 01:54:42 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Thu, 14 Jan 2016 01:54:40 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9+ mfc-stable10+ X-Bugzilla-Changed-Fields: bug_status flagtypes.name resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2016 01:54:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Progress |Closed Flags|mfc-stable9?, mfc-stable10? |mfc-stable9+, mfc-stable10+ Resolution|--- |FIXED --- Comment #7 from Pedro F. Giffuni --- Committed and MFCd. Thanks! --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 14 01:58:04 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4F87A8272D for ; Thu, 14 Jan 2016 01:58:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 94E3B14B2 for ; Thu, 14 Jan 2016 01:58:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0E1w4dL088048 for ; Thu, 14 Jan 2016 01:58:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206059] [ext2fs][patch] EXT4: cannot mount filesystems < 512 MiB in size: "ext2fs: no space for extra inode timestamps" Date: Thu, 14 Jan 2016 01:58:04 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Works As Intended X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2016 01:58:04 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206059 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |Works As Intended Status|New |Closed --- Comment #2 from Pedro F. Giffuni --- As explained, I think we are doing the right thing here. --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 14 09:29:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75430A6DA1B for ; Thu, 14 Jan 2016 09:29:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3AD019B2 for ; Thu, 14 Jan 2016 09:29:40 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u0E9TZYh078173 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 14 Jan 2016 11:29:35 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u0E9TZYh078173 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u0E9TYWS078172; Thu, 14 Jan 2016 11:29:34 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 14 Jan 2016 11:29:34 +0200 From: Konstantin Belousov To: Rick Macklem Cc: FreeBSD Filesystems Subject: Re: panic ffs_truncate3 (maybe fuse being evil) Message-ID: <20160114092934.GL72455@kib.kiev.ua> References: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> <20160110154518.GU3625@kib.kiev.ua> <1773157955.158922767.1452698181137.JavaMail.zimbra@uoguelph.ca> <1351730674.159022044.1452699617235.JavaMail.zimbra@uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1351730674.159022044.1452699617235.JavaMail.zimbra@uoguelph.ca> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2016 09:29:41 -0000 On Wed, Jan 13, 2016 at 10:40:17AM -0500, Rick Macklem wrote: > > Well, although I have r174973 in the kernel that crashes, it looks like this > > bug might have been around for a while. > > Here's what I've figured out sofar. > > 1 - The crashes only occur if soft updates are disabled. This isn't > > surprising > > if you look at ffs_truncate(), since the test for the panic isn't done > > when soft updates are enabled. > > Here's the snippet from ffs_truncate(), in case you are interested: > > if (DOINGSOFTDEP(vp)) { > > 335 if (softdeptrunc == 0 && journaltrunc == 0) { > > 336 /* > > 337 * If a file is only partially truncated, then > > 338 * we have to clean up the data structures > > 339 * describing the allocation past the truncation > > 340 * point. Finding and deallocating those > > structures > > 341 * is a lot of work. Since partial truncation > > occurs > > 342 * rarely, we solve the problem by syncing the > > file > > 343 * so that it will have no data structures left. > > 344 */ > > 345 if ((error = ffs_syncvnode(vp, MNT_WAIT, 0)) != > > 0) > > 346 return (error); > > 347 } else { > > 348 flags = IO_NORMAL | (needextclean ? IO_EXT: 0); > > 349 if (journaltrunc) > > 350 softdep_journal_freeblocks(ip, cred, > > length, > > 351 flags); > > 352 else > > 353 softdep_setup_freeblocks(ip, length, > > flags); > > 354 ASSERT_VOP_LOCKED(vp, "ffs_truncate1"); > > 355 if (journaltrunc == 0) { > > 356 ip->i_flag |= IN_CHANGE | IN_UPDATE; > > 357 error = ffs_update(vp, 0); > > 358 } > > 359 return (error); > > 360 } > > 361 } > > You can see that it always returns once in this code block. The only way the > > code can get > > past this block if soft updates are enabled is a "goto extclean;", which > > takes you past > > the "panic()". I see. Indeed, 'new' SU(+J) code invalidates and frees buffers in ffs_softdep.c:indir_trunc(). Due to the nature of the SU code, which attaches dependencies to the buffers, it is somewhat problematic to assert same condition, since buffers might survive longer due to unsatisfied dependencies. > > > > By adding a few printf()s, I have determined: > > - The bo_clean.bv_cnt == 1 when the panic occurs and the b_lblkno of the > > buffer is -ve. What is the exact value of lblkno ? There are two kinds of buffers with negative lblk which may exist on the UFS vnode queues. One is indirect block, and another is the metadata block. Metadata lblks are -1 and -2, indir blocks numbers are calculated by formula you can see in calculation of indir_lbn[] in ffs_truncate(). > > > > If you look at vtruncbuf(): > > trunclbn = (length + blksize - 1) / blksize; > > 1726 > > 1727 ASSERT_VOP_LOCKED(vp, "vtruncbuf"); > > 1728 restart: > > 1729 bo = &vp->v_bufobj; > > 1730 BO_LOCK(bo); > > 1731 anyfreed = 1; > > 1732 for (;anyfreed;) { > > 1733 anyfreed = 0; > > 1734 TAILQ_FOREACH_SAFE(bp, &bo->bo_clean.bv_hd, b_bobufs, > > nbp) { > > 1735 if (bp->b_lblkno < trunclbn) > > 1736 continue; > > When length == 0 --> trunclbn is 0, but the test at line#1735 will skip over > > the b_lblkno > > because it is negative. > > > > That is as far as I've gotten. A couple of things I need help from others on: > > - Is vtruncbuf() skipping over the cases where b_lblkno < 0 a feature or a > > bug? It is a feature, since vtruncbuf() intent is truncation of the normal file data. Negative lbns are used as a trick to manage metadata or other data not normally accessible to the read/write syscalls, but which still is reasonable to consider inode data. > > - If it is a feature, then what needs to be done in the code after the > > vtruncbuf() > > call in ffs_truncate() to ensure the buffer is gone by the time the panic > > check is > > done? > > --> I do see a bunch of code after the vtruncbuf() call related to indirect > > blocks > > (which I think use the -ve b_lblkno?), but I'll admit I don't understand > > it well > > enough to know if it expects vtruncbuf() to leave the -ve block on the > > bo_hd list? Either the left blocks are extattrs, and then the assert should be improved, or we trully leak buffers and the leak must be fixed. We will see by the block number. > > > > Obviously fixing vtruncbuf() to get rid of these -ve b_lblkno entries would > > be easy, > > but I don't know if that is a feature or a bug? > > > > I did look at the commit logs and vtruncbuf() has been like this for at least > > 10years. > > (I can only guess very few run UFS without soft updates or others would see > > these panic()s.) > > > > I am now running with soft updates enabled to avoid the crashes, but I can > > easily test any > > patch if others can a patch to try. > > > Oh, and one more thing. > Maybe having the buffer for an indirect block hanging off the vnode at the > end of ffs_truncate() to 0 length is ok. After all, this is happening in > VOP_INACTIVE() and the vnode isn't being recycled yet? > (ie. The panic() test is not needed?) See above, depends on kind of the block. But ufs_inactive() truncating the inode means that the nlinks went to zero, and the inode is going to be freed. So we better not to leak anything. From owner-freebsd-fs@freebsd.org Thu Jan 14 11:09:13 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37214A8109B for ; Thu, 14 Jan 2016 11:09:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 27A871D58 for ; Thu, 14 Jan 2016 11:09:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0EB9DnW073496 for ; Thu, 14 Jan 2016 11:09:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Thu, 14 Jan 2016 11:09:13 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: feature, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: nowak@tepeserwery.pl X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2016 11:09:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 nowak@tepeserwery.pl changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nowak@tepeserwery.pl --- Comment #15 from nowak@tepeserwery.pl --- (In reply to Damjan Jovanovic from comment #0) There is "zero_region" that can be used here instead of the static array. while (xfersize > 0) { ssize_t len =3D MIN(xfersize, ZERO_REGION_SIZE); error =3D uiomove(__DECONST(void *, zero_region), len, uio); if (error) { return error; } xfersize -=3D len; } --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Fri Jan 15 02:27:26 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B2B1A828A5 for ; Fri, 15 Jan 2016 02:27:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5B8771C7D for ; Fri, 15 Jan 2016 02:27:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0F2ROhL096391 for ; Fri, 15 Jan 2016 02:27:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206238] FUSE can't enter to DIRECT_IO mode during file create. Date: Fri, 15 Jan 2016 02:27:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2016 02:27:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206238 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jan 15 03:14:27 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A310A83C1C for ; Fri, 15 Jan 2016 03:14:27 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 476F51D21 for ; Fri, 15 Jan 2016 03:14:26 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:gQlelxAOhcJp7W+nnrwsUyQJP3N1i/DPJgcQr6AfoPdwSP79p8bcNUDSrc9gkEXOFd2CrakU1ayH6+u9CSQp2tWojjMrSNR0TRgLiMEbzUQLIfWuLgnFFsPsdDEwB89YVVVorDmROElRH9viNRWJ+iXhpQAbFhi3DwdpPOO9QteU1JTpkbHqsMCDKyxzxxODIppKZC2sqgvQssREyaBDEY0WjiXzn31TZu5NznlpL1/A1zz158O34YIxu38I46FppIZ8VvDZeKIjUbVeEDUge0o44Mr2rh7dBV+M4WAAU2Ycnx5gDA3M7RW8VZD05HjUrO14jRObNs6+aLk/WjCv6u8/UhrhgyQDOjsR7WbYl8F0lKIdqxv39E83+JLdfIzAbKk2RajaZ95PHWc= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CpBABEY5hW/61jaINehH+IUrULhg8CgXgRAQEBAQEBAQGBCYItggcBAQEDASMEUgULAgEIDgoCAg0ZAgJXAgSINAiwSJBBAQEBAQEFAQEBAQEBHYEAhVWDe4EEhFWDH4FJBY45iF2POYdvhTREiiKDbwI4LIIRHIF7IIVdgQgBAQE X-IronPort-AV: E=Sophos;i="5.22,297,1449550800"; d="scan'208";a="261346736" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 14 Jan 2016 22:14:19 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 29EF115F56E; Thu, 14 Jan 2016 22:14:19 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id X-SVGYUzdJ5g; Thu, 14 Jan 2016 22:14:18 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 769E015F571; Thu, 14 Jan 2016 22:14:18 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id iaL4zGTuuJ1h; Thu, 14 Jan 2016 22:14:18 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 5D5D315F56E; Thu, 14 Jan 2016 22:14:18 -0500 (EST) Date: Thu, 14 Jan 2016 22:14:18 -0500 (EST) From: Rick Macklem To: Konstantin Belousov Cc: FreeBSD Filesystems Message-ID: <964333498.161527381.1452827658163.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20160114092934.GL72455@kib.kiev.ua> References: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> <20160110154518.GU3625@kib.kiev.ua> <1773157955.158922767.1452698181137.JavaMail.zimbra@uoguelph.ca> <1351730674.159022044.1452699617235.JavaMail.zimbra@uoguelph.ca> <20160114092934.GL72455@kib.kiev.ua> Subject: Re: panic ffs_truncate3 (maybe fuse being evil) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: panic ffs_truncate3 (maybe fuse being evil) Thread-Index: 80nFT1BwAZhJG0p7JfyYPX23NdQnyQ== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2016 03:14:27 -0000 Kostik wrote: > On Wed, Jan 13, 2016 at 10:40:17AM -0500, Rick Macklem wrote: > > > Well, although I have r174973 in the kernel that crashes, it looks like > > > this > > > bug might have been around for a while. > > > Here's what I've figured out sofar. > > > 1 - The crashes only occur if soft updates are disabled. This isn't > > > surprising > > > if you look at ffs_truncate(), since the test for the panic isn't > > > done > > > when soft updates are enabled. > > > Here's the snippet from ffs_truncate(), in case you are interested: > > > if (DOINGSOFTDEP(vp)) { > > > 335 if (softdeptrunc == 0 && journaltrunc == 0) { > > > 336 /* > > > 337 * If a file is only partially truncated, > > > then > > > 338 * we have to clean up the data structures > > > 339 * describing the allocation past the > > > truncation > > > 340 * point. Finding and deallocating those > > > structures > > > 341 * is a lot of work. Since partial > > > truncation > > > occurs > > > 342 * rarely, we solve the problem by syncing > > > the > > > file > > > 343 * so that it will have no data structures > > > left. > > > 344 */ > > > 345 if ((error = ffs_syncvnode(vp, MNT_WAIT, 0)) > > > != > > > 0) > > > 346 return (error); > > > 347 } else { > > > 348 flags = IO_NORMAL | (needextclean ? IO_EXT: > > > 0); > > > 349 if (journaltrunc) > > > 350 softdep_journal_freeblocks(ip, cred, > > > length, > > > 351 flags); > > > 352 else > > > 353 softdep_setup_freeblocks(ip, length, > > > flags); > > > 354 ASSERT_VOP_LOCKED(vp, "ffs_truncate1"); > > > 355 if (journaltrunc == 0) { > > > 356 ip->i_flag |= IN_CHANGE | IN_UPDATE; > > > 357 error = ffs_update(vp, 0); > > > 358 } > > > 359 return (error); > > > 360 } > > > 361 } > > > You can see that it always returns once in this code block. The only way > > > the > > > code can get > > > past this block if soft updates are enabled is a "goto extclean;", which > > > takes you past > > > the "panic()". > I see. Indeed, 'new' SU(+J) code invalidates and frees buffers in > ffs_softdep.c:indir_trunc(). Due to the nature of the SU code, which > attaches dependencies to the buffers, it is somewhat problematic to > assert same condition, since buffers might survive longer due to > unsatisfied dependencies. > > > > > > > By adding a few printf()s, I have determined: > > > - The bo_clean.bv_cnt == 1 when the panic occurs and the b_lblkno of the > > > buffer is -ve. > What is the exact value of lblkno ? > It was -1 for the crash I looked at. (Since all are the same backtrace and happen for the same test sequence, I suspect they all are.) > There are two kinds of buffers with negative lblk which may exist on > the UFS vnode queues. One is indirect block, and another is the metadata > block. Metadata lblks are -1 and -2, indir blocks numbers are calculated > by formula you can see in calculation of indir_lbn[] in ffs_truncate(). > Thanks. I knew indirect blocks were -ve block #s, but I didn't know about the metadata ones. > > > > > > If you look at vtruncbuf(): > > > trunclbn = (length + blksize - 1) / blksize; > > > 1726 > > > 1727 ASSERT_VOP_LOCKED(vp, "vtruncbuf"); > > > 1728 restart: > > > 1729 bo = &vp->v_bufobj; > > > 1730 BO_LOCK(bo); > > > 1731 anyfreed = 1; > > > 1732 for (;anyfreed;) { > > > 1733 anyfreed = 0; > > > 1734 TAILQ_FOREACH_SAFE(bp, &bo->bo_clean.bv_hd, > > > b_bobufs, > > > nbp) { > > > 1735 if (bp->b_lblkno < trunclbn) > > > 1736 continue; > > > When length == 0 --> trunclbn is 0, but the test at line#1735 will skip > > > over > > > the b_lblkno > > > because it is negative. > > > > > > That is as far as I've gotten. A couple of things I need help from others > > > on: > > > - Is vtruncbuf() skipping over the cases where b_lblkno < 0 a feature or > > > a > > > bug? > It is a feature, since vtruncbuf() intent is truncation of the normal > file data. Negative lbns are used as a trick to manage metadata or other > data not normally accessible to the read/write syscalls, but which still > is reasonable to consider inode data. > > > > - If it is a feature, then what needs to be done in the code after the > > > vtruncbuf() > > > call in ffs_truncate() to ensure the buffer is gone by the time the > > > panic > > > check is > > > done? > > > --> I do see a bunch of code after the vtruncbuf() call related to > > > indirect > > > blocks > > > (which I think use the -ve b_lblkno?), but I'll admit I don't > > > understand > > > it well > > > enough to know if it expects vtruncbuf() to leave the -ve block on > > > the > > > bo_hd list? > Either the left blocks are extattrs, and then the assert should be improved, > or we trully leak buffers and the leak must be fixed. We will see by > the block number. > So, if I've understood you correctly, a -1 b_lblkno is an extattr one and the assert should allow it and not panic? I can change the assert, although it will end up about 6 lines of code;-) If you think I should do this, I will do so and test it. > > > > > > Obviously fixing vtruncbuf() to get rid of these -ve b_lblkno entries > > > would > > > be easy, > > > but I don't know if that is a feature or a bug? > > > > > > I did look at the commit logs and vtruncbuf() has been like this for at > > > least > > > 10years. > > > (I can only guess very few run UFS without soft updates or others would > > > see > > > these panic()s.) > > > > > > I am now running with soft updates enabled to avoid the crashes, but I > > > can > > > easily test any > > > patch if others can a patch to try. > > > > > Oh, and one more thing. > > Maybe having the buffer for an indirect block hanging off the vnode at the > > end of ffs_truncate() to 0 length is ok. After all, this is happening in > > VOP_INACTIVE() and the vnode isn't being recycled yet? > > (ie. The panic() test is not needed?) > See above, depends on kind of the block. > > But ufs_inactive() truncating the inode means that the nlinks went to zero, > and the inode is going to be freed. So we better not to leak anything. > Righto. Thanks for your help, rick From owner-freebsd-fs@freebsd.org Fri Jan 15 04:05:01 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8DB4A839DD for ; Fri, 15 Jan 2016 04:05:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C9BB911BB for ; Fri, 15 Jan 2016 04:05:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0F451Zl078305 for ; Fri, 15 Jan 2016 04:05:01 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206238] FUSE can't enter to DIRECT_IO mode during file create. Date: Fri, 15 Jan 2016 04:05:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rmacklem@uoguelph.ca X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2016 04:05:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206238 rmacklem@uoguelph.ca changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rmacklem@uoguelph.ca --- Comment #1 from rmacklem@uoguelph.ca --- Are you referring to cases where the MooseFS fuse file system sets FOPEN_DIRECT_IO in the reply to a fuse open? or Are you referring to processes on FreeBSD that open with O_DIRECT? Btw, if you look at PR#194293, I have a patch there that forces DIRECT_IO for the case of WRONLY opens, so that a small buffered write doesn't get into trouble when the buffer cache code tries to read the entire block in. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jan 15 09:58:01 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E80DA84B64 for ; Fri, 15 Jan 2016 09:58:01 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 174A412D6 for ; Fri, 15 Jan 2016 09:57:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u0F9voAQ040493 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 15 Jan 2016 11:57:50 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u0F9voAQ040493 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u0F9vnrp040492; Fri, 15 Jan 2016 11:57:49 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 15 Jan 2016 11:57:49 +0200 From: Konstantin Belousov To: Rick Macklem Cc: FreeBSD Filesystems Subject: Re: panic ffs_truncate3 (maybe fuse being evil) Message-ID: <20160115095749.GC3942@kib.kiev.ua> References: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> <20160110154518.GU3625@kib.kiev.ua> <1773157955.158922767.1452698181137.JavaMail.zimbra@uoguelph.ca> <1351730674.159022044.1452699617235.JavaMail.zimbra@uoguelph.ca> <20160114092934.GL72455@kib.kiev.ua> <964333498.161527381.1452827658163.JavaMail.zimbra@uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <964333498.161527381.1452827658163.JavaMail.zimbra@uoguelph.ca> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2016 09:58:01 -0000 On Thu, Jan 14, 2016 at 10:14:18PM -0500, Rick Macklem wrote: > Kostik wrote: > > What is the exact value of lblkno ? > > > It was -1 for the crash I looked at. (Since all are the same backtrace and > happen for the same test sequence, I suspect they all are.) > > > There are two kinds of buffers with negative lblk which may exist on > > the UFS vnode queues. One is indirect block, and another is the metadata > > block. Metadata lblks are -1 and -2, indir blocks numbers are calculated > > by formula you can see in calculation of indir_lbn[] in ffs_truncate(). > > > Thanks. I knew indirect blocks were -ve block #s, but I didn't know about > the metadata ones. s/metadata/ext attributes data/ Sorry. > So, if I've understood you correctly, a -1 b_lblkno is an extattr one and > the assert should allow it and not panic? Yes. > > I can change the assert, although it will end up about 6 lines of code;-) > If you think I should do this, I will do so and test it. It is under INVARIANTS anyway. I do see a value in checking that truncation is consistent. We did have many bugs there, which usually caused user data loss. Some caution is worth it, IMO. From owner-freebsd-fs@freebsd.org Fri Jan 15 15:20:17 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B4EFA83691 for ; Fri, 15 Jan 2016 15:20:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6C2251A21 for ; Fri, 15 Jan 2016 15:20:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0FFKHFJ026255 for ; Fri, 15 Jan 2016 15:20:17 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205708] Kernel panic possibly happening while adding/removing/listing ZFS snapshot Date: Fri, 15 Jan 2016 15:20:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: karl@denninger.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2016 15:20:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205708 karl@denninger.net changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |karl@denninger.net --- Comment #6 from karl@denninger.net --- Are you running the patch from here? https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D187594 If not are you willing to try the latest version of that patch (from 2015-12-10)? I have had a somewhat-related crash going on for quite a while, and while i= t is difficult to reproduce I've had it happen both with earlier versions of the above patch and on a "clean" system. HOWEVER, the last version has now been running since early December -- with no panics. My blowups were all related to snapshot manipulation during backup operatio= ns, so this may be related. If so then the root is likely the result of a modu= lo extension that returns an unexpected delay product and causes trouble elsew= here in the code. The latest version of that patch prevents the extension from occurring. I can pull out just that couple of lines of code that appear to have been effective in stopping the blowups if you'd like to try something simpler and leave the rest of the ZFS code alone. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jan 15 17:25:55 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4F1EA8356A for ; Fri, 15 Jan 2016 17:25:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9642C14AA for ; Fri, 15 Jan 2016 17:25:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0FHPtwv024801 for ; Fri, 15 Jan 2016 17:25:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205708] Kernel panic possibly happening while adding/removing/listing ZFS snapshot Date: Fri, 15 Jan 2016 17:25:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: root_dfjo1hpy@meo.ws X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2016 17:25:55 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205708 --- Comment #7 from Fusl --- Hi, sorry for not updating this earlier. I upgraded the memory of this machine from 36GB (5*4GB + 4*GB modules) to 7= 2GB (9*8GB) memory and it seems to be more stable now (not a single issue since then): [root@chernobyl:~] uptime 5:17PM up 14 days, 2:39, 5 users, load averages: 0.23, 0.24, 0.30 [root@chernobyl:~] sysctl vfs.zfs.l2arc_noprefetch vfs.zfs.l2arc_noprefetch: 0 [root@chernobyl:~] zfs get -Hovalue sync,logbias data disabled throughput Since this is a production environment I would rather like to keep the syst= em running like that. However, if the system happens to crash another time I w= ill try to update and/or apply the patch. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jan 16 02:00:14 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA12DA83372 for ; Sat, 16 Jan 2016 02:00:14 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 8B4651F76 for ; Sat, 16 Jan 2016 02:00:13 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:rkslZhysT5RSPmrXCy+O+j09IxM/srCxBDY+r6Qd0eIUIJqq85mqBkHD//Il1AaPBtWFraMewLGH+4nbGkU+or+5+EgYd5JNUxJXwe43pCcHRPC/NEvgMfTxZDY7FskRHHVs/nW8LFQHUJ2mPw6anHS+4HYoFwnlMkItf6KuStCU3p78jLH60qaQSjsLrQL1Wal1IhSyoFeZnegtqqwmFJwMzADUqGBDYeVcyDAgD1uSmxHh+pX4p8Y7oGx48sgs/M9YUKj8Y79wDfkBVGxnYCgJ45jIsh/YUQKJ+noaGk8blh1VGA/bpEX5WYztsyjwu+5V1yyTPMmwRrcxD2eM9aBuHSXpgyRPEjcy82Xaj4QklqdSqxGlqhlX3onbfYyRLPo4daqLLoBSfnZIQssED38JOYi7dYZaSrNZZes= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2B6AgB+o5lW/61jaINehH+IULMrDoFjhg8CgWkUAQEBAQEBAQGBCYItggcBAQEDASMEUhACAQgOChEZAgICVQIEiCYIsCCQQgEBAQEBAQEBAgEBAQEBAQEBAREJhlWEf4RUGYMHgUkFjjmIYIJ3jEWNI0SKKINvAiABQ4IRHIF7IIYjgQgBAQE X-IronPort-AV: E=Sophos;i="5.22,302,1449550800"; d="scan'208";a="263139087" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 15 Jan 2016 21:00:06 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 7B87C15F565; Fri, 15 Jan 2016 21:00:06 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id D9p7abzjxw1t; Fri, 15 Jan 2016 21:00:06 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 165B415F56D; Fri, 15 Jan 2016 21:00:06 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id CGatm6dkpcES; Fri, 15 Jan 2016 21:00:05 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id ECDB315F565; Fri, 15 Jan 2016 21:00:05 -0500 (EST) Date: Fri, 15 Jan 2016 21:00:05 -0500 (EST) From: Rick Macklem To: Konstantin Belousov Cc: FreeBSD Filesystems , Kirk McKusick Message-ID: <1817287612.162823118.1452909605928.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20160115095749.GC3942@kib.kiev.ua> References: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> <20160110154518.GU3625@kib.kiev.ua> <1773157955.158922767.1452698181137.JavaMail.zimbra@uoguelph.ca> <1351730674.159022044.1452699617235.JavaMail.zimbra@uoguelph.ca> <20160114092934.GL72455@kib.kiev.ua> <964333498.161527381.1452827658163.JavaMail.zimbra@uoguelph.ca> <20160115095749.GC3942@kib.kiev.ua> Subject: Re: panic ffs_truncate3 (maybe fuse being evil) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_162823116_2090676742.1452909605926" X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: panic ffs_truncate3 (maybe fuse being evil) Thread-Index: jkQM2w6o2xU5AkSgXtYhq7afE/taqA== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2016 02:00:14 -0000 ------=_Part_162823116_2090676742.1452909605926 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Kostik wrote: > On Thu, Jan 14, 2016 at 10:14:18PM -0500, Rick Macklem wrote: > > Kostik wrote: > > > What is the exact value of lblkno ? > > > > > It was -1 for the crash I looked at. (Since all are the same backtrace and > > happen for the same test sequence, I suspect they all are.) > > > > > There are two kinds of buffers with negative lblk which may exist on > > > the UFS vnode queues. One is indirect block, and another is the metadata > > > block. Metadata lblks are -1 and -2, indir blocks numbers are calculated > > > by formula you can see in calculation of indir_lbn[] in ffs_truncate(). > > > > > Thanks. I knew indirect blocks were -ve block #s, but I didn't know about > > the metadata ones. > s/metadata/ext attributes data/ > > Sorry. > No problem. Extended attributes are metadata and I suspect there might be other metadata handled this way someday. > > So, if I've understood you correctly, a -1 b_lblkno is an extattr one and > > the assert should allow it and not panic? > Yes. > > > > > I can change the assert, although it will end up about 6 lines of code;-) > > If you think I should do this, I will do so and test it. > It is under INVARIANTS anyway. I do see a value in checking that truncation > is consistent. We did have many bugs there, which usually caused user data > loss. Some caution is worth it, IMO. > > I've attached the patch with the modified test for the panic. I ran a test with the old one (with the panic changed to a printf) just before this one and the printf happened, but no panic. After the test, the file system fsck'd ok, so I think this might be ok. I've added Kirk as a cc. Maybe he can review this? I can't do commits until mid-April, so maybe you can commit it? (I don't think there is any rush, since almost everyone will be using soft updates. To be honest, I have no idea why I turned off soft updates at some point for this filesystem.) Thanks for your help, rick ------=_Part_162823116_2090676742.1452909605926 Content-Type: text/x-patch; name=ffsinode.patch Content-Disposition: attachment; filename=ffsinode.patch Content-Transfer-Encoding: base64 LS0tIHVmcy9mZnMvZmZzX2lub2RlLmMuc2F2CTIwMTYtMDEtMTAgMjA6MTE6NDYuNDA2NjgyMDAw IC0wNTAwCisrKyB1ZnMvZmZzL2Zmc19pbm9kZS5jCTIwMTYtMDEtMTUgMTc6MjI6NTguNDY1OTkx MDAwIC0wNTAwCkBAIC01NDQsNyArNTQ0LDEyIEBAIGRvbmU6CiAJQk9fTE9DSyhibyk7CiAJaWYg KGxlbmd0aCA9PSAwICYmCiAJICAgIChmcy0+ZnNfbWFnaWMgIT0gRlNfVUZTMl9NQUdJQyB8fCBp cC0+aV9kaW4yLT5kaV9leHRzaXplID09IDApICYmCi0JICAgIChiby0+Ym9fZGlydHkuYnZfY250 ID4gMCB8fCBiby0+Ym9fY2xlYW4uYnZfY250ID4gMCkpCisJICAgICgoYm8tPmJvX2RpcnR5LmJ2 X2NudCA+IDAgJiYgKFRBSUxRX0VNUFRZKCZiby0+Ym9fZGlydHkuYnZfaGQpIHx8CisJICAgICBU QUlMUV9GSVJTVCgmYm8tPmJvX2RpcnR5LmJ2X2hkKS0+Yl9sYmxrbm8gPj0gMCB8fAorCSAgICAg VEFJTFFfRklSU1QoJmJvLT5ib19kaXJ0eS5idl9oZCktPmJfbGJsa25vIDwgLTIpKSB8fAorCSAg ICAgKGJvLT5ib19jbGVhbi5idl9jbnQgPiAwICYmIChUQUlMUV9FTVBUWSgmYm8tPmJvX2NsZWFu LmJ2X2hkKSB8fAorCSAgICAgVEFJTFFfRklSU1QoJmJvLT5ib19jbGVhbi5idl9oZCktPmJfbGJs a25vID49IDAgfHwKKwkgICAgIFRBSUxRX0ZJUlNUKCZiby0+Ym9fY2xlYW4uYnZfaGQpLT5iX2xi bGtubyA8IC0yKSkpKQogCQlwYW5pYygiZmZzX3RydW5jYXRlMyIpOwogCUJPX1VOTE9DSyhibyk7 CiAjZW5kaWYgLyogSU5WQVJJQU5UUyAqLwo= ------=_Part_162823116_2090676742.1452909605926-- From owner-freebsd-fs@freebsd.org Sat Jan 16 19:11:24 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 10D54A85C60 for ; Sat, 16 Jan 2016 19:11:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 806A31E28 for ; Sat, 16 Jan 2016 19:11:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u0GJBHPO077942 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 16 Jan 2016 21:11:17 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u0GJBHPO077942 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u0GJBGId077940; Sat, 16 Jan 2016 21:11:16 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 16 Jan 2016 21:11:16 +0200 From: Konstantin Belousov To: Rick Macklem Cc: FreeBSD Filesystems , Kirk McKusick Subject: Re: panic ffs_truncate3 (maybe fuse being evil) Message-ID: <20160116191116.GI3942@kib.kiev.ua> References: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> <20160110154518.GU3625@kib.kiev.ua> <1773157955.158922767.1452698181137.JavaMail.zimbra@uoguelph.ca> <1351730674.159022044.1452699617235.JavaMail.zimbra@uoguelph.ca> <20160114092934.GL72455@kib.kiev.ua> <964333498.161527381.1452827658163.JavaMail.zimbra@uoguelph.ca> <20160115095749.GC3942@kib.kiev.ua> <1817287612.162823118.1452909605928.JavaMail.zimbra@uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1817287612.162823118.1452909605928.JavaMail.zimbra@uoguelph.ca> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2016 19:11:24 -0000 On Fri, Jan 15, 2016 at 09:00:05PM -0500, Rick Macklem wrote: > --- ufs/ffs/ffs_inode.c.sav 2016-01-10 20:11:46.406682000 -0500 > +++ ufs/ffs/ffs_inode.c 2016-01-15 17:22:58.465991000 -0500 > @@ -544,7 +544,12 @@ done: > BO_LOCK(bo); > if (length == 0 && > (fs->fs_magic != FS_UFS2_MAGIC || ip->i_din2->di_extsize == 0) && > - (bo->bo_dirty.bv_cnt > 0 || bo->bo_clean.bv_cnt > 0)) > + ((bo->bo_dirty.bv_cnt > 0 && (TAILQ_EMPTY(&bo->bo_dirty.bv_hd) || > + TAILQ_FIRST(&bo->bo_dirty.bv_hd)->b_lblkno >= 0 || > + TAILQ_FIRST(&bo->bo_dirty.bv_hd)->b_lblkno < -2)) || > + (bo->bo_clean.bv_cnt > 0 && (TAILQ_EMPTY(&bo->bo_clean.bv_hd) || > + TAILQ_FIRST(&bo->bo_clean.bv_hd)->b_lblkno >= 0 || > + TAILQ_FIRST(&bo->bo_clean.bv_hd)->b_lblkno < -2)))) > panic("ffs_truncate3"); > BO_UNLOCK(bo); > #endif /* INVARIANTS */ So I tried to rewrite the assert to be more readable, and noted a thing which I should have noted much earlier. The ip->i_din2->di_extsize == 0 test ensures that both extattr blocks are deallocated. In other words, even with the new information about lba of the leaked buffers, the assert is still correct, it catched stray buffer which should not be there. Was IO_EXT flag passed to the ffs_truncate() invocation where the assert ffs_truncate3 fired ? From owner-freebsd-fs@freebsd.org Sat Jan 16 23:20:34 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07D1FA85355 for ; Sat, 16 Jan 2016 23:20:34 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id AA8DF1567 for ; Sat, 16 Jan 2016 23:20:33 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:RfAOiRf58O09C9LXdmtrKkawlGMj4u6mDksu8pMizoh2WeGdxc6yYB7h7PlgxGXEQZ/co6odzbGG7eaxAidZvcvJmUtBWaIPfidNsd8RkQ0kDZzNImzAB9muURYHGt9fXkRu5XCxPBsdMs//Y1rPvi/6tmZKSV3BPAZ4bt74BpTVx5zukbvipduCOE4R2Gf1SIgxBSv1hD2ZjtMRj4pmJ/R54TryiVwMRd5rw3h1L0mYhRf265T41pdi9yNNp6BprJYYAu3SNp41Rr1ADTkgL3t9pIiy7UGCHkOz4S4/VWMNjhNOHwnDpDv3WpDsqSzk/r5+3zKGPM78QLQcVjGr7qMtQxjt3nQpLTk8pVvWgc84qatQoxasolQr2Yvda4KROf9WY6TSYN4eXWoHVc8HBH8JOZ+1c4ZaV7lJBu1ftYSo4gJW9RY= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DOAQA7z5pW/61jaINehH+IULMvAQ2BY4YPAoFZFAEBAQEBAQEBgQmCLYIIAQEEIwRSEAIBCA4KAgINGQICVwIEiC6vMJAGAQEBAQEBAQECAQEBAQEBAQEbgQCFVYR/h3SBSQWOOYhhnGBEjhcCIAEBQoQpIIZLgQgBAQE X-IronPort-AV: E=Sophos;i="5.22,306,1449550800"; d="scan'208";a="263227470" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 16 Jan 2016 18:20:32 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 8A59F15F55D; Sat, 16 Jan 2016 18:20:32 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id P7GTd2spEJii; Sat, 16 Jan 2016 18:20:32 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 11E8115F565; Sat, 16 Jan 2016 18:20:32 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id D6Q9EhYKCbMH; Sat, 16 Jan 2016 18:20:31 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id EA0D615F55D; Sat, 16 Jan 2016 18:20:31 -0500 (EST) Date: Sat, 16 Jan 2016 18:20:31 -0500 (EST) From: Rick Macklem To: Konstantin Belousov Cc: FreeBSD Filesystems , Kirk McKusick Message-ID: <853868666.163292727.1452986431921.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20160116191116.GI3942@kib.kiev.ua> References: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca> <1773157955.158922767.1452698181137.JavaMail.zimbra@uoguelph.ca> <1351730674.159022044.1452699617235.JavaMail.zimbra@uoguelph.ca> <20160114092934.GL72455@kib.kiev.ua> <964333498.161527381.1452827658163.JavaMail.zimbra@uoguelph.ca> <20160115095749.GC3942@kib.kiev.ua> <1817287612.162823118.1452909605928.JavaMail.zimbra@uoguelph.ca> <20160116191116.GI3942@kib.kiev.ua> Subject: Re: panic ffs_truncate3 (maybe fuse being evil) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: panic ffs_truncate3 (maybe fuse being evil) Thread-Index: vlYzyP6+/0R7IKjPIHye/djPI/6TZw== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2016 23:20:34 -0000 Kostik wrote: > On Fri, Jan 15, 2016 at 09:00:05PM -0500, Rick Macklem wrote: > > --- ufs/ffs/ffs_inode.c.sav 2016-01-10 20:11:46.406682000 -0500 > > +++ ufs/ffs/ffs_inode.c 2016-01-15 17:22:58.465991000 -0500 > > @@ -544,7 +544,12 @@ done: > > BO_LOCK(bo); > > if (length == 0 && > > (fs->fs_magic != FS_UFS2_MAGIC || ip->i_din2->di_extsize == 0) && > > - (bo->bo_dirty.bv_cnt > 0 || bo->bo_clean.bv_cnt > 0)) > > + ((bo->bo_dirty.bv_cnt > 0 && (TAILQ_EMPTY(&bo->bo_dirty.bv_hd) || > > + TAILQ_FIRST(&bo->bo_dirty.bv_hd)->b_lblkno >= 0 || > > + TAILQ_FIRST(&bo->bo_dirty.bv_hd)->b_lblkno < -2)) || > > + (bo->bo_clean.bv_cnt > 0 && (TAILQ_EMPTY(&bo->bo_clean.bv_hd) || > > + TAILQ_FIRST(&bo->bo_clean.bv_hd)->b_lblkno >= 0 || > > + TAILQ_FIRST(&bo->bo_clean.bv_hd)->b_lblkno < -2)))) > > panic("ffs_truncate3"); > > BO_UNLOCK(bo); > > #endif /* INVARIANTS */ > > So I tried to rewrite the assert to be more readable, and noted a thing > which I should have noted much earlier. > > The ip->i_din2->di_extsize == 0 test ensures that both extattr blocks > are deallocated. In other words, even with the new information about > lba of the leaked buffers, the assert is still correct, it catched > stray buffer which should not be there. > > Was IO_EXT flag passed to the ffs_truncate() invocation where the > assert ffs_truncate3 fired ? > Yes. The only call to UFS_TRUNCATE() in ufs_inactive() specified both IO_EXT | IO_NORMAL. rick From owner-freebsd-fs@freebsd.org Sat Jan 16 23:22:04 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1541BA853E0 for ; Sat, 16 Jan 2016 23:22:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 03CE9185C for ; Sat, 16 Jan 2016 23:22:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0GNM2hv028503 for ; Sat, 16 Jan 2016 23:22:03 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206238] FUSE can't enter to DIRECT_IO mode during file create. Date: Sat, 16 Jan 2016 23:22:02 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rmacklem@uoguelph.ca X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2016 23:22:04 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206238 --- Comment #2 from rmacklem@uoguelph.ca --- Created attachment 165690 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D165690&action= =3Dedit patch fuse so it forces DIRECT_IO for WRONLY opens --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jan 16 23:24:51 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A473A85674 for ; Sat, 16 Jan 2016 23:24:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0C3C61AD3 for ; Sat, 16 Jan 2016 23:24:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0GNOoxh036811 for ; Sat, 16 Jan 2016 23:24:50 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206238] FUSE can't enter to DIRECT_IO mode during file create. Date: Sat, 16 Jan 2016 23:24:50 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rmacklem@uoguelph.ca X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2016 23:24:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206238 --- Comment #3 from rmacklem@uoguelph.ca --- Created attachment 165691 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D165691&action= =3Dedit patch fuse so that a create with DIRECT_IO opens with DIRECT_IO Please try applying both of these patches in order. The second one fixes your problem, I think? (Assuming it was a create with FOPEN_DIRECT_IO in its reply.) The first is from PR#194293 and forces DIRECT_IO when a file is WRONLY opened. Please let me know if this fixes your problem. --=20 You are receiving this mail because: You are the assignee for the bug.=