From owner-freebsd-stable@freebsd.org Tue May 7 13:03:06 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8CAF15854B3 for ; Tue, 7 May 2019 13:03:05 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "gromit.dlib.vt.edu", Issuer "Chumby Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D2F61886E9 for ; Tue, 7 May 2019 13:03:02 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from mather.gromit23.net (c-98-244-101-97.hsd1.va.comcast.net [98.244.101.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gromit.dlib.vt.edu (Postfix) with ESMTPSA id 7CD5CF6; Tue, 7 May 2019 09:03:01 -0400 (EDT) Content-Type: text/plain; charset=utf-8; delsp=yes; format=flowed Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: ZFS... From: Paul Mather In-Reply-To: Date: Tue, 7 May 2019 09:03:00 -0400 Cc: freebsd-stable Content-Transfer-Encoding: 8bit Message-Id: <26B407D8-3EED-47CA-81F6-A706CF424567@gromit.dlib.vt.edu> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <20190430102024.E84286@mulder.mintsol.com> <41FA461B-40AE-4D34-B280-214B5C5868B5@punkt.de> <20190506080804.Y87441@mulder.mintsol.com> <08E46EBF-154F-4670-B411-482DCE6F395D@sorbs.net> <33D7EFC4-5C15-4FE0-970B-E6034EF80BEF@gromit.dlib.vt.edu> To: Michelle Sullivan X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: D2F61886E9 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dmarc=fail reason="" header.from=vt.edu (policy=none) X-Spamd-Result: default: False [-2.97 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[vt.edu : No valid SPF, No valid DKIM,none]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-0.94)[ip: (-2.40), ipnet: 128.173.0.0/16(-1.20), asn: 1312(-1.05), country: US(-0.06)]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: chumby.dlib.vt.edu]; RCPT_COUNT_TWO(0.00)[2]; SUBJ_ALL_CAPS(0.45)[6]; R_SPF_NA(0.00)[]; NEURAL_HAM_SHORT(-0.96)[-0.963,0]; RECEIVED_SPAMHAUS_PBL(0.00)[97.101.244.98.zen.spamhaus.org : 127.0.0.10]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:1312, ipnet:128.173.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 May 2019 13:03:06 -0000 On May 7, 2019, at 1:02 AM, Michelle Sullivan wrote: >> On 07 May 2019, at 10:53, Paul Mather wrote: >> >>> On May 6, 2019, at 10:14 AM, Michelle Sullivan >>> wrote: >>> >>> My issue here (and not really what the blog is about) FreeBSD is >>> defaulting to it. >> >> You've said this at least twice now in this thread so I'm assuming >> you're asserting it to be true. >> >> As of FreeBSD 12.0-RELEASE (and all earlier releases), FreeBSD does NOT >> default to ZFS. >> >> The images distributed by freebsd.org, e.g., Vagrant boxes, ARM images, >> EC2 instances, etc., contain disk images where FreeBSD resides on UFS. >> For example, here's what you end up with when you launch a 12.0-RELEASE >> instance using defaults on AWS (us-east-1 region: ami-03b0f822e17669866): >> >> root@freebsd:/usr/home/ec2-user # gpart show >> => 3 20971509 ada0 GPT (10G) >> 3 123 1 freebsd-boot (62K) >> 126 20971386 2 freebsd-ufs (10G) >> >> And this is what you get when you "vagrant up" the >> freebsd/FreeBSD-12.0-RELEASE box: >> >> root@freebsd:/home/vagrant # gpart show >> => 3 65013755 ada0 GPT (31G) >> 3 123 1 freebsd-boot (62K) >> 126 2097152 2 freebsd-swap (1.0G) >> 2097278 62914560 3 freebsd-ufs (30G) >> 65011838 1920 - free - (960K) >> >> >> When you install from the 12.0-RELEASE ISO, the first option listed >> during the partitioning stage is "Auto (UFS) Guided Disk Setup". The >> last option listed---after "Open a shell and partition by hand" is "Auto >> (ZFS) Guided Root-on-ZFS". In other words, you have to skip over UFS >> and manual partitioning to select the ZFS install option. >> >> So, I don't see what evidence there is that FreeBSD is defaulting to >> ZFS. It hasn't up to now. Will FreeBSD 13 default to ZFS? > > Umm.. well I install by memory stick images and I had a 10.2 and an 11.0 > both of which had root on zfs as the default.. I had to manually change > them. I haven’t looked at anything later... so did something change? Am > I in cloud cookoo land? I don't know about that, but you may well be misremembering. I just pulled down the 10.2 and 11.0 installers from http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/old-releases and in both cases the choices listed in the "Partitioning" step are the same as in the current 12.0 installer: "Auto (UFS) Guided Disk Setup" is listed first and selected by default. "Auto (ZFS) Guided Root-on-ZFS" is listed last (you have to skip past other options such as manually partitioning by hand to select it). I'm confident in saying that ZFS is (or was) not the default partitioning option in either 10.2 or 11.0 as officially released by FreeBSD. Did you use a custom installer you made yourself when installing 10.2 or 11.0? > >>> FreeBSD used to be targeted at enterprise and devs (which is where I >>> found it)... however the last few years have been a big push into the >>> consumer (compete with Linux) market.. so you have an OS that concerns >>> itself with the desktop and upgrade after upgrade after upgrade (not >>> just patching security issues, but upgrades as well.. just like windows >>> and OSX)... I get it.. the money is in the keeping of the user base.. >>> but then you install a file system which is dangerous on a single disk >>> by default... dangerous because it’s trusted and “can’t fail” .. until >>> it goes titsup.com and then the entire drive is lost and all the data >>> on it.. it’s the double standard... advocate you need ECC ram, >>> multiple vdevs etc, then single drive it.. sorry.. which one is it? >>> Gaaaaaarrrrrrrgggghhhhhhh! >> >> >> As people have pointed out elsewhere in this thread, it's false to claim >> that ZFS is unsafe on consumer hardware. It's no less safe than UFS on >> single-disk setups. >> >> Because anecdote is not evidence, I will refrain from saying, "I've lost >> far more data on UFS than I have on ZFS (especially when SUJ was shaking >> out its bugs)..." >;-) >> >> What I will agree with is that, probably due to its relative youth, ZFS >> has less forensics/data recovery tools than UFS. I'm sure this will >> improve as time goes on. (I even posted a link to an article describing >> someone adding ZFS support to a forensics toolkit earlier in this >> thread.) > > The problem I see with that statement is that the zfs dev mailing lists > constantly and consistently following the line of, the data is always > right there is no need for a “fsck” (which I actually get) but it’s used > to shut down every thread... the irony is I’m now installing windows 7 > and SP1 on a usb stick (well it’s actually installed, but sp1 isn’t > finished yet) so I can install a zfs data recovery tool which reports to > be able to “walk the data” to retrieve all the files... the irony eh... > install windows7 on a usb stick to recover a FreeBSD installed zfs > filesystem... will let you know if the tool works, but as it was > recommended by a dev I’m hopeful... have another array (with zfs I might > add) loaded and ready to go... if the data recovery is successful I’ll > blow away the original machine and work out what OS and drive setup will > be safe for the data in the future. I might even put FreeBSD and zfs > back on it, but if I do it won’t be in the current Zraid2 config. There is no more irony in installing a data recovery tool to recover a trashed ZFS pool than there is in installing one to recover a trashed UFS file system. No file system is bulletproof, which is why everyone I know recommends a backup/disaster recovery strategy commensurate with the value you place on your data. There WILL be some combination of events that will lead to irretrievable data loss. Your extraordinary sequence of mishaps apparently met the threshold for ZFS on your setup. I don't see how any of this leads to the conclusion that ZFS is "dangerous" to use as a file system. What I believe is dangerous is relying on a post-mortem crash data recovery methodology as a substitute for a backup strategy for data that, in hindsight, is considered important enough to keep. No matter how resilient ZFS or UFS may be, they are no substitute for backups when it comes to data you care about. (File system resiliency will not protect you, e.g., from Ransomware or other malicious or accidental acts of data destruction.) Cheers, Paul.