From owner-freebsd-fs@freebsd.org Mon Nov 12 07:30:31 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 518F411330D1 for ; Mon, 12 Nov 2018 07:30:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id C15CA81A0A for ; Mon, 12 Nov 2018 07:30:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 82A7F11330D0; Mon, 12 Nov 2018 07:30:30 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EDDD11330CF for ; Mon, 12 Nov 2018 07:30:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D127181A09 for ; Mon, 12 Nov 2018 07:30:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id E70A213A7D for ; Mon, 12 Nov 2018 07:30:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id wAC7USRo063586 for ; Mon, 12 Nov 2018 07:30:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id wAC7USrG063585 for fs@FreeBSD.org; Mon, 12 Nov 2018 07:30:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 195485] [ufs] mksnap_ffs(8) cannot create snapshot with journaled soft updates enabled Date: Mon, 12 Nov 2018 07:30:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: C15CA81A0A X-Spamd-Result: default: False [-105.89 / 200.00]; FORGED_RECIPIENTS_FORWARDING(0.00)[]; ALLOW_DOMAIN_WHITELIST(-100.00)[freebsd.org]; FORWARDED(0.00)[fs@mailman.ysv.freebsd.org]; SPF_FAIL_FORWARDING(0.00)[]; TO_DN_NONE(0.00)[]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; XAW_SERVICE_ACCT(1.00)[]; RCVD_IN_DNSWL_MED(-0.20)[5.0.0.0.0.5.0.0.0.0.0.0.0.0.0.0.a.6.0.2.4.5.2.2.0.0.9.1.1.0.0.2.list.dnswl.org : 127.0.9.2]; MX_GOOD(-0.01)[cached: mx66.freebsd.org]; NEURAL_HAM_SHORT(-1.00)[-0.999,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; IP_SCORE(-3.68)[ip: (-9.84), ipnet: 2001:1900:2254::/48(-4.77), asn: 10310(-3.70), country: US(-0.09)]; ASN(0.00)[asn:10310, ipnet:2001:1900:2254::/48, country:US]; FORGED_RECIPIENTS(0.00)[fs@FreeBSD.org,freebsd-fs@freebsd.org]; TO_DOM_EQ_FROM_DOM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_NO_DN(0.00)[]; RCVD_COUNT_SEVEN(0.00)[7] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Nov 2018 07:30:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D195485 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Open CC| |mckusick@FreeBSD.org --- Comment #2 from Kirk McKusick --- (In reply to t_uemura from comment #1) Short answer: snapshots work while SU+J is running. The problem arises beca= use the fsck code that does the journal recovery does not know how to repair snapshots. Thus after a crash recovery all the snapshots that were on the filesystem are possibly corrupted and will cause a panic if used. Long answer: when files are deleted, the blocks are normally returned to the list of free blocks so that they can be allocated to new files. When a filesystem contains snapshots, each freed blocks is first offered to each of the snapshots so that they can claim it if it is part of one of the files in the snapshot. By claiming a block they prevent it from being put on the lis= t of free blocks and thus its contents will be preserved for the snapshotted fil= e. The journal recovery code has never had the logic added to it to do these checks. Hence, when it frees blocks, it does not check the snapshots to see= if they want to claim these blocks. Thus blocks that should be claimed by the snapshots are instead put on the list of free blocks and will eventually be reused. If one of these blocks is part of the metadata of a file in a snaps= hot (such as a block of indirect pointers) and that block gets overwritten with other data, then attempts to access that file in the snapshot will cause a = data inconsistency leading to a kernel panic. The correct solution is to extract the code from the kernel that handles freeing of blocks and add it to the journal recovery code in fsck. This is a lot of complicated code and would take a lot of effort to do. As ZFS provid= es cheap snapshots, that is the filesystem of preference for folks that want snapshot functionality. The only remaining use for snapshots in UFS is the ability to do live dumps. Thus I have not been motivated to go to the effo= rt to migrate the kernel code to fsck (and nobody has offered to pay me the $2= 5K to have me do it). An easier solution would be to simply delete all the snapshots as part of d= oing the filesystem recovery. The problem is that while there is a list of all t= he inode numbers for the active snapshots in the superblock, we do not know the pathnames for all of these snapshots, so we would have to do a complete traversal of the filesystem to find them which would largely negate the spe= ed benefit of journaling. Another easy solution would be to truncate all the snapshots to zero length= and stop offering them as snapshots. This would be much quicker as we have the = list of inode number that need to be truncated and all we would be left to clean= up would be a list of zero-length files which could be handled by a find after= the system is up and running. I am happy to review changes if someone wants to implement this solution (or the more difficult correct solution noted above). --=20 You are receiving this mail because: You are the assignee for the bug.=