From owner-freebsd-fs@freebsd.org Sun Dec 30 21:01:23 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D57DC1434D32 for ; Sun, 30 Dec 2018 21:01:22 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 71A0B6DABF for ; Sun, 30 Dec 2018 21:01:22 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 3006F1434D31; Sun, 30 Dec 2018 21:01:22 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E96C31434D2F for ; Sun, 30 Dec 2018 21:01:21 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 883556DAB5 for ; Sun, 30 Dec 2018 21:01:21 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 9CA16FCCC for ; Sun, 30 Dec 2018 21:01:20 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id wBUL1KFW027783 for ; Sun, 30 Dec 2018 21:01:20 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id wBUL1KTh027777 for fs@FreeBSD.org; Sun, 30 Dec 2018 21:01:20 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201812302101.wBUL1KTh027777@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: fs@FreeBSD.org Subject: Problem reports for fs@FreeBSD.org that need special attention Date: Sun, 30 Dec 2018 21:01:20 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Dec 2018 21:01:23 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic New | 217062 | for file systems mounted with -o noexec, exec=off Open | 140068 | [smbfs] [patch] smbfs does not allow semicolon in Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 211491 | System hangs after "Uptime" on reboot with ZFS Open | 221909 | [ZFS] Add a sysctl to toggle send_corrupt_data 6 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Wed Jan 2 20:44:02 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0464142FDA6 for ; Wed, 2 Jan 2019 20:44:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 357516F7AC for ; Wed, 2 Jan 2019 20:44:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id E9F1B142FDA1; Wed, 2 Jan 2019 20:44:01 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D883C142FDA0 for ; Wed, 2 Jan 2019 20:44:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B3576F7A7 for ; Wed, 2 Jan 2019 20:44:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C279B17615 for ; Wed, 2 Jan 2019 20:44:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x02Ki0Zt015966 for ; Wed, 2 Jan 2019 20:44:00 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x02Ki0Dx015965 for fs@FreeBSD.org; Wed, 2 Jan 2019 20:44:00 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 234581] [fusefs] no VOP_ADVLOCK support Date: Wed, 02 Jan 2019 20:44:00 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cem@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jan 2019 20:44:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D234581 Conrad Meyer changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Jan 2 22:06:59 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 622541432CEB for ; Wed, 2 Jan 2019 22:06:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id F18EE72C01 for ; Wed, 2 Jan 2019 22:06:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id B4ACF1432CE9; Wed, 2 Jan 2019 22:06:58 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A24F91432CE8 for ; Wed, 2 Jan 2019 22:06:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41C9E72BFE for ; Wed, 2 Jan 2019 22:06:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 80DA5181DC for ; Wed, 2 Jan 2019 22:06:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x02M6vfb013678 for ; Wed, 2 Jan 2019 22:06:57 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x02M6vWq013676 for fs@FreeBSD.org; Wed, 2 Jan 2019 22:06:57 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 233415] 12.0-RC1 - sysutils/fusefs-exfat broken on 12.x Date: Wed, 02 Jan 2019 22:06:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: asomers@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: samm@os2.kiev.ua X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jan 2019 22:06:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D233415 Alan Somers changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|fs@FreeBSD.org |samm@os2.kiev.ua CC| |asomers@FreeBSD.org --- Comment #2 from Alan Somers --- I doubt it's a general fuse problem. Assigning to port maintainer. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 3 10:34:40 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A1D71424481 for ; Thu, 3 Jan 2019 10:34:39 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1069B6EDD8 for ; Thu, 3 Jan 2019 10:34:36 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.5] (unknown [192.148.167.11]) by proxypop01.sare.net (Postfix) with ESMTPA id 8F1899DD8FE for ; Thu, 3 Jan 2019 11:34:26 +0100 (CET) From: Borja Marcos Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Interesting: ZFS scrub prefetch hurting sequential scrub performance? Message-Id: <8ECF7513-9DFB-46EF-86BA-DB717D713792@sarenet.es> Date: Thu, 3 Jan 2019 11:34:25 +0100 To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 1069B6EDD8 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of borjam@sarenet.es designates 195.16.151.151 as permitted sender) smtp.mailfrom=borjam@sarenet.es X-Spamd-Result: default: False [2.94 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.16.150.0/23]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain,multipart/related]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_MEDIUM(0.68)[0.676,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MX_GOOD(-0.01)[smtp.sarenet.es,smtp.sarenet.es,smtp.sarenet.es]; NEURAL_SPAM_LONG(1.00)[1.000,0]; RCVD_IN_DNSWL_NONE(0.00)[151.151.16.195.list.dnswl.org : 127.0.10.0]; NEURAL_SPAM_SHORT(0.06)[0.061,0]; SUBJECT_ENDS_QUESTION(1.00)[]; DMARC_NA(0.00)[sarenet.es]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; ASN(0.00)[asn:3262, ipnet:195.16.128.0/19, country:ES]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.01)[country: ES(0.05)]; RCVD_COUNT_TWO(0.00)[2] X-Mailman-Approved-At: Thu, 03 Jan 2019 11:34:08 +0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jan 2019 10:34:40 -0000 Hi, I have noticed that my scrubs have become painfully slow. I am wondering = wether I=E2=80=99ve just hit some worst case or maybe there is some interaction between the ZFS sequential scrub and scrub = prefetch. I don=E2=80=99t recall seeing this behavior before the sequential scrub code was committed.=20 Did I hit some worst case or should scrub prefetch be disabled with the = new sequential scrub code? # zpool status pool: pool state: ONLINE scan: scrub in progress since Sat Dec 29 03:56:02 2018 133G scanned at 309K/s, 129G issued at 300K/s, 619G total 0 repaired, 20.80% done, no estimated completion time When this happened last month I tried rebooting the server and = restarting the scrub and everything went better.=20 The first graph shows the disk I/O bandwith history for the last week. = When the scrub started disk I/O =E2=80=9Cbusy percent=E2=80=9D reached almost 100 %. And curiously the transfer rates looked rather = healthy at around 10 MBps of read activity. At first I suspected a misbehaving disk slowing down the whole process = with retries but all the disks show a similar service time pattern. One attached for reference. Looking at the rest of the stats for some misbehavior hints I saw = arctats_prefetch_metadata misses raising to about 2000 per second and arcstats_l2_misses following the same pattern.=20= Could it be prefetch spending a lot of time writing on the l2arc only to = have the data evicted due to misses? I have tried disabling scrub prefetch (vfs.zfs.no_scrub_prefetch=3D1) = and, voila! everything picked up speed. Now with a zpool iostat I see bursts of 100+ MBps reading activity and a = proper scrub activity. Disk busy percent has gone down to around 50% and cache stats have = become much better. Turns out that most of the I/O activity was just pointless writes to the L2ARC. Now, the hardware configuration. The server has only 8 GB of memory with a maximum configured ARC size of = 4 GB.=20 It has a LSI2008 card with IR firmware. I didn=C2=B4t bother to cross = flash but anyway I am not using the RAID facilities, it=C2=B4s just configured like a plain HBA. mps0: port 0x9000-0x90ff mem = 0xdfff0000-0xdfffffff,0xdff80000-0xdffbffff irq 17 at device 0.0 = numa-domain 0 on pci4 mps0: Firmware: 20.00.07.00, Driver: 21.02.00.00-fbsd mps0: IOCCapabilities: = 185c zpool status pool: pool state: ONLINE scan: scrub in progress since Sat Dec 29 03:56:02 2018 323G scanned at 742K/s, 274G issued at 632K/s, 619G total 0 repaired, 44.32% done, no estimated completion time config: NAME STATE READ WRITE CKSUM pool ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da12 ONLINE 0 0 0 da13 ONLINE 0 0 0 da14 ONLINE 0 0 0 da9 ONLINE 0 0 0 da15 ONLINE 0 0 0 da3 ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 da10 ONLINE 0 0 0 da4 ONLINE 0 0 0 da5 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 da8 ONLINE 0 0 0 logs da11p2 ONLINE 0 0 0 cache da11p3 ONLINE 0 0 0 errors: No known data errors Yes, both ZIL and L2ARC on the same disk (a SSD). I know it=E2=80=99s = not optimal but I guess it=E2=80=99s better than the high latency of conventional disks, # camcontrol devlist at scbus6 target 11 lun 0 (pass0,da0) at scbus6 target 15 lun 0 (pass1,da1) at scbus6 target 17 lun 0 (pass2,da2) at scbus6 target 18 lun 0 (pass3,da3) at scbus6 target 20 lun 0 (pass4,da4) at scbus6 target 21 lun 0 (pass5,da5) at scbus6 target 22 lun 0 (pass6,da6) at scbus6 target 23 lun 0 (pass7,da7) at scbus6 target 24 lun 0 (pass8,da8) at scbus6 target 25 lun 0 (pass9,da9) at scbus6 target 26 lun 0 = (pass10,da10) at scbus6 target 27 lun 0 = (ses0,pass11) at scbus6 target 28 lun 0 = (pass12,da11) at scbus6 target 29 lun 0 = (pass13,da12) at scbus6 target 30 lun 0 = (pass14,da13) at scbus6 target 32 lun 0 = (pass15,da14) at scbus6 target 33 lun 0 = (pass16,da15) at scbus13 target 0 lun 0 = (pass17,cd0) Hope the attachments reach the list, otherwise I will mail them to = anyone interested. Cheers, Borja. From owner-freebsd-fs@freebsd.org Thu Jan 3 22:38:04 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 074DF1439E0A for ; Thu, 3 Jan 2019 22:38:04 +0000 (UTC) (envelope-from jay@sentenai.com) Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 75A3C6CAF9 for ; Thu, 3 Jan 2019 22:38:02 +0000 (UTC) (envelope-from jay@sentenai.com) Received: by mail-qk1-x72c.google.com with SMTP id w204so20597001qka.2 for ; Thu, 03 Jan 2019 14:38:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sentenai-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Zhksf6/x5L2KGMp1XdR+VNijaHfESDe7pR/sAJlqj9c=; b=0BzEi0A2qeJWf75vLW/sIHyfC+7YQh/N1s4k/PrVnyIqZ+Jw7Ja2owvsYg1TBGoZkB 8NJthBot+Zu+B8cF81XLkRLj/nUxXjSVpB/eV/G2wBvpeeeDWfoKVeyQUdmaQGVEATOn WGiKgUVLGFdl/YmXBARbb/sekK9iD2+hlQ+Hlk8GoAdUz6dDAXW346pes600oqBmbppC Ic4OC4wowdO50NOb6RwEJrjkGkZmV8ouA9nLKxbPjh0KpO9y8iKFeWW9cr9PXu/i0Gdl fM4ZvP5Kh59fzeKzCiEdoSDms7JoxzF5aCNFMfZ9Scz0fO/dpAl3gV9oEP9eu4sLPe7I R9PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Zhksf6/x5L2KGMp1XdR+VNijaHfESDe7pR/sAJlqj9c=; b=pqtdpSemktbl75bxfqjZvFk3WvC7F1bLvGcWdaTzeVE2r2oIYZPVGCBwCA5KLiPVUC IoyQ6z/8miZGwHxoRseTgQO5YmeVGe/ufXQJMSCb6RMqUPaQ6XwT0VQE1eZg6DC/CSb0 +4/51xMzXtssG9YIKHoMNU/Wb1jKnbeaCng35WfceW9UEcdNmrDs7Fx7KGq4Zshuu1a5 S4iWxHk+6HAj01mPUrg5fIgCBmZcFfwHUDo5Uy8HpiE8e6jV6KUTcznrCZKB0dsw0d0v nEW/5qzHwzpnGPdLDxgoqN9AthVQzWLN16KxJ0/7OaeQ4/xSCl6d0hpzvBY+L0iI5nSy TyzA== X-Gm-Message-State: AJcUukcFcdq8FbNx5JlVdRv43xfZizdmg2OcMW3Qg4E3fmxi4tjuXSm5 LYS5Iu/GRkne6bbO0flAQ+/nVr9rmWmm0xacnHac X-Google-Smtp-Source: ALg8bN4rvJiJsNMIomsj9Xw5YhWLQwf17fiYY3d4nvxKoRRWkcgQG21pO4J6o8K+7ASy680Iz+3Z6nslKDF3deHGSGw= X-Received: by 2002:a37:ac5:: with SMTP id 188mr45561822qkk.135.1546555081657; Thu, 03 Jan 2019 14:38:01 -0800 (PST) MIME-Version: 1.0 References: <01000167dd7b16cd-58ee733f-c662-42d5-9c75-2301fbc8377a-000000@email.amazonses.com> <01000167f27b4bf1-86cf2251-ed4f-4f6f-88d7-9f2969338a57-000000@email.amazonses.com> In-Reply-To: <01000167f27b4bf1-86cf2251-ed4f-4f6f-88d7-9f2969338a57-000000@email.amazonses.com> From: Jay Edwards Date: Thu, 3 Jan 2019 17:37:50 -0500 Message-ID: Subject: Re: Any plans for EC2 AMIs with RootOnZFS? To: Colin Percival Cc: Ben Woods , freebsd-cloud@freebsd.org, "freebsd-fs@freebsd.org" X-Rspamd-Queue-Id: 75A3C6CAF9 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=sentenai-com.20150623.gappssmtp.com header.s=20150623 header.b=0BzEi0A2; spf=pass (mx1.freebsd.org: domain of jay@sentenai.com designates 2607:f8b0:4864:20::72c as permitted sender) smtp.mailfrom=jay@sentenai.com X-Spamd-Result: default: False [-4.12 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; URI_COUNT_ODD(1.00)[5]; DKIM_TRACE(0.00)[sentenai-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,alt4.aspmx.l.google.com,alt3.aspmx.l.google.com,alt2.aspmx.l.google.com,aspmx.l.google.com]; NEURAL_HAM_SHORT(-0.94)[-0.939,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MIME_TRACE(0.00)[0:+,1:+]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.995,0]; R_DKIM_ALLOW(-0.20)[sentenai-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[sentenai.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[c.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.67)[ip: (-9.50), ipnet: 2607:f8b0::/32(-2.15), asn: 15169(-1.64), country: US(-0.08)]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jan 2019 22:38:04 -0000 I've got a fairly robust collection of utilities for Google Cloud that: 1) Starts up an official FreeBSD image 2) Mounts a separate drive (the ZFS target) 3) Installs the release; installs all of the various GCP packages 4) Configures a bunch of custom stuff applicable to me 5) Creates an image that is then used as a base for any further custom images. It shouldn't be that difficult to add AWS support. Jay Edwards On Thu, Dec 27, 2018 at 8:48 PM Colin Percival wrote: > [Adding freebsd-fs in the hope of finding more ZFS people.] > > On 12/23/18 3:51 PM, Colin Percival wrote: > > On 12/23/18 7:12 AM, Ben Woods wrote: > >> Is there any plans to provide official FreeBSD AMIs in the AWS > Marketplace > >> which use RootOnZFS out of the box? > > > > Nothing concrete yet, but it's on my radar. Now that FreeBSD AMIs are > being > > published by a separate (release engineering) account, I'm more > comfortable > > with working on more "experimental" AMIs. (And if this turns out to be > widely > > useful I'll talk to the release engineering team about adding it to the > set > > they publish.) > > I've created an experimental FreeBSD 12.0 ZFS AMI in the us-east-1 region: > ami-0786f5b55d5aa573f > > Since I'm basically a ZFS newbie (I run it on my laptop, but only with the > default setup from the installer, and I've never used any of the fancy ZFS > features) I don't know if I've set everything up properly. So far I have > one piece of feedback, which is that I should have marked canmount=off for > zroot. > > Can people please try out this AMI and let me know if there's anything else > (ZFS-related) which I should fix? Some time next week I'll fix whatever > people notice and build AMIs for all the regions. > > -- > Colin Percival > Security Officer Emeritus, FreeBSD | The power to serve > Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid > _______________________________________________ > freebsd-cloud@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-cloud > To unsubscribe, send any mail to "freebsd-cloud-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Fri Jan 4 10:52:51 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 966A0142598A for ; Fri, 4 Jan 2019 10:52:51 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu1176c.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 744E08AE9D for ; Fri, 4 Jan 2019 10:52:50 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.5] (unknown [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPA id 05C289DC4E3; Fri, 4 Jan 2019 11:52:39 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: Interesting: ZFS scrub prefetch hurting sequential scrub performance? From: Borja Marcos In-Reply-To: <8ECF7513-9DFB-46EF-86BA-DB717D713792@sarenet.es> Date: Fri, 4 Jan 2019 11:52:37 +0100 Cc: freebsd-fs@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <8ECF7513-9DFB-46EF-86BA-DB717D713792@sarenet.es> To: Borja Marcos X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 744E08AE9D X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of borjam@sarenet.es designates 195.16.148.151 as permitted sender) smtp.mailfrom=borjam@sarenet.es X-Spamd-Result: default: False [-0.66 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.63)[-0.631,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.16.148.0/24]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[sarenet.es]; NEURAL_HAM_LONG(-0.79)[-0.791,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[smtp.sarenet.es,smtp.sarenet.es,smtp.sarenet.es]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[151.148.16.195.list.dnswl.org : 127.0.10.0]; NEURAL_HAM_SHORT(-0.44)[-0.441,0]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3262, ipnet:195.16.128.0/19, country:ES]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.01)[country: ES(0.05)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2019 10:52:51 -0000 > On 3 Jan 2019, at 11:34, Borja Marcos wrote: >=20 >=20 > Hi, >=20 > I have noticed that my scrubs have become painfully slow. I am = wondering wether I=E2=80=99ve just hit some worst case or maybe > there is some interaction between the ZFS sequential scrub and scrub = prefetch. I don=E2=80=99t recall seeing this behavior > before the sequential scrub code was committed.=20 >=20 > Did I hit some worst case or should scrub prefetch be disabled with = the new sequential scrub code? I have done a test with the old scrub code (vfs.zfs.zfs_scan_legacy=3D1) = and I see a very similar behavior, with the=20 scrub stalling again. Once more, disabling prefetch for the scrub = (vfs.zfs.no_scrub_prefetch=3D1) solves the issue. I suffered this problem on 11 at some point but I attributed it = (wrongly!) to hardware problems at the time. Not I=E2=80=99ve just found a talk about a new prefetch mechanism for = the scrub by Tom Caputi. Could it be the problem? https://www.youtube.com/watch?v=3Dupn9tYh917s Borja. From owner-freebsd-fs@freebsd.org Fri Jan 4 16:28:44 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DED0D1430B99 for ; Fri, 4 Jan 2019 16:28:43 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 073166F99F for ; Fri, 4 Jan 2019 16:28:42 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x829.google.com with SMTP id u47so36303350qtj.6 for ; Fri, 04 Jan 2019 08:28:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=EpIkyrPkR4bWZrXqtMAw1HJS39ZWXwBIk3S8EYGU5gc=; b=fFsZfk81lHt/Cy/BpOWPFp+ciKM1/GXa003jEbg3ohl02LqG1gO+Vcp217F870c2kf Yi00DJ0MDBGgC/7r/r2roteflub4cgHbgiHBogAlRCvv+gDNHY9aZ62bvioGpjcqcMk5 K6cu/E3eqVrmeQm8hRjf5j7v7l5koI0zAn0uH+GLiqnIEJ8IR2ZMDumZBiAl4KOn3sHB s5ppK3gyBT11OuP9b2MH1CVaUxI8F7CdKDqT+RjblmTiMAv9xPRl4IIXlPZjKqN/z4Np n403OCSDQZWm9WiHCfr/ybB7SzwTZJPzPbaKjB/EStRc1A2eI/T3X8pld06p/te5ZFFk NBAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=EpIkyrPkR4bWZrXqtMAw1HJS39ZWXwBIk3S8EYGU5gc=; b=DI18rPD8H9vyjv2GxdQ1T0SsfC71ZM6rRUtqL4xfFoJ0Yu4n7j6hYc64uYh3sRkPle H1qnqMEiFt/CHU9wULyoXA38d7z4vXMjIutaHC08nJc+VKqJ/qt6TrNU6w+zVENmpRSX aSb5rcCUbxS4lflHe2/eF2b7B1zdttp/r87HbYAra4FxhCRoz3Xqlx0bUAeanY3Iy3mw Hn25NTcCBz421ufoP38s4toDgZB25loHMuH7Pcb1USFwmpuoYiqbsI9Fj17/pw5MrCM1 xZD+Z+nWoRWPcsMc73XS6zcqaYMXO6M7ETiMtBJxdc/uPLaiM7bf/z3RWlOhzk1Zd7Bh H3Fg== X-Gm-Message-State: AA+aEWYL77UU5umc5g5pQTBfoEkDca0CAoXXNiwwvUpS2GrBsAlDm9vk Kp5eqLcdMdaXL+onBvPK2Wm+a/SSxspmPPNVDV7/Sg== X-Google-Smtp-Source: AFSGD/XSHQyQgEviajp/TYVr5AJhDoJ3DvoPFBd6TT/jy+W5bjd4wCVGRuqridnURK8Xl+LtlwKJ0E7CgZKBdiiQVgY= X-Received: by 2002:ac8:3f2d:: with SMTP id c42mr50755766qtk.33.1546619322157; Fri, 04 Jan 2019 08:28:42 -0800 (PST) MIME-Version: 1.0 References: <8ECF7513-9DFB-46EF-86BA-DB717D713792@sarenet.es> In-Reply-To: From: Warner Losh Date: Fri, 4 Jan 2019 09:28:31 -0700 Message-ID: Subject: Re: Interesting: ZFS scrub prefetch hurting sequential scrub performance? To: Borja Marcos Cc: FreeBSD FS X-Rspamd-Queue-Id: 073166F99F X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=fFsZfk81 X-Spamd-Result: default: False [-4.56 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.972,0]; IP_SCORE(-2.59)[ip: (-9.02), ipnet: 2607:f8b0::/32(-2.17), asn: 15169(-1.66), country: US(-0.08)]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[9.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: ALT1.aspmx.l.google.com]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2019 16:28:44 -0000 On Fri, Jan 4, 2019 at 3:53 AM Borja Marcos wrote: > > > > On 3 Jan 2019, at 11:34, Borja Marcos wrote: > > > > > > Hi, > > > > I have noticed that my scrubs have become painfully slow. I am wonderin= g > wether I=E2=80=99ve just hit some worst case or maybe > > there is some interaction between the ZFS sequential scrub and scrub > prefetch. I don=E2=80=99t recall seeing this behavior > > before the sequential scrub code was committed. > > > > Did I hit some worst case or should scrub prefetch be disabled with the > new sequential scrub code? > > I have done a test with the old scrub code (vfs.zfs.zfs_scan_legacy=3D1) = and > I see a very similar behavior, with the > scrub stalling again. > > Once more, disabling prefetch for the scrub (vfs.zfs.no_scrub_prefetch=3D= 1) > solves the issue. > > I suffered this problem on 11 at some point but I attributed it (wrongly!= ) > to hardware problems at the time. > > Not I=E2=80=99ve just found a talk about a new prefetch mechanism for the= scrub by > Tom Caputi. Could it be the problem? > https://www.youtube.com/watch?v=3Dupn9tYh917s It's always been a hard problem to schedule background activity without affecting foreground performance. For Hard Drives this isn't so terrible to do: keep the queue depths small so that when any new work arrives, the latency in switching between the two workloads is small. With SSDs, it gets harder, though in a read-only workload it degenerates to about the same. SSDs do their own read ahead, sometimes, and they have lots of background activity that can be triggered by reads (like if it could read the block, but the error rate from the NAND was over some threshold, the drive might decide to copy all the data out of that block because data with that error rate won't be readable with the correction codes in place long enough to meet the retention specs). And writes can also trigger this background behavior. So switching between the foreground and background tasks becomes even more sluggish. But I think in ZFS' case, it may just be a bit of a bug in backing off the scrub operation to allow better local host performance... Warner From owner-freebsd-fs@freebsd.org Sat Jan 5 00:05:06 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC757143F4D0 for ; Sat, 5 Jan 2019 00:05:05 +0000 (UTC) (envelope-from bilbo@hobbiton.org) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B5308AE46 for ; Sat, 5 Jan 2019 00:05:03 +0000 (UTC) (envelope-from bilbo@hobbiton.org) Received: by mail-qt1-x836.google.com with SMTP id t13so42183367qtn.3 for ; Fri, 04 Jan 2019 16:05:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ofwilsoncreek-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=OM0abPyZXm3oXQCUtkwH8nF6gsg/MkIw6cnEk0eDh6g=; b=PSAgNn8bBvS13NjeOJNRKEpn+xsLswd8tmAZ8YWgPmB3PCSfrDSDxri4+suCwTkSCx TOdaHX+05KEcTHlkk7fl7QsmP/qMyrQS7c96BJSWSE4rBAbhgG+sgS/eHeByW9Uiiyec fYhAMYVFN+P0lmgGXBW71CwvWnQky0r4YMlVw1y7g+NwIS0Iqpgw1Z4u6YUZaIiwa67h 22mE/LFUK/H3svULjUeBRW2B4CdgGTTvrg3ZzAKkjtGA7c5ky+nKlK8CyzkbkLvoObzo nHExhEs9zmi0Tm9ndae8pY4b5updY8kW3bQOwxF0+K2QpFzkIex7i5UhZ20PWAG0b2vO hHCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=OM0abPyZXm3oXQCUtkwH8nF6gsg/MkIw6cnEk0eDh6g=; b=moTAhYDsz04EvyQasBtDY+aA2/+YPoLXwfkvmqKLlf69OnRPqn30oD3W+yOW3kxbk5 xzzMQP6/pJcsgeSMFw9H4Z4PIyNKWLb0IimRa5HEubgZ8ofV2eCHc9pGM9nHgi8n9D28 q50SIVyAiZuFo4C1SEk6wMI5gQ3sF2aZl4EesHx/NFwzEbXgu4A7Zh+xJhWl4kvOEH6b UbmFVQNFfb8Ym0LzmHgU5jrVJfFbxqLSe+SBYmVTCzgxdRGrtG9l72AYzNhAHTI2UoGA 7BsdB0E1XFRhzTcwn6tIlPMij3soW5X7DPz4bHY6NNYbAG/7nNK/vjxQDmOtTgksEBnu UpkA== X-Gm-Message-State: AA+aEWat2qBAmMKOrIhav4prXNBUexFPw4uotJzWGQsZ1uE8t1uODcQy faQgqougm44Lcv5y3GgUNbnbg+dTd5HsNrnGFrlVXh7S X-Google-Smtp-Source: ALg8bN7obtbxcb2z/MpYhZSIefMHCjmK1Qq84mGo3KKhWn7JP4GjjRT4pOI84BDiOrbKNXHnmkkADzgh9fh8+AZQ8yo= X-Received: by 2002:ac8:1941:: with SMTP id g1mr50088190qtk.193.1546646703098; Fri, 04 Jan 2019 16:05:03 -0800 (PST) MIME-Version: 1.0 References: <01000167dd7b16cd-58ee733f-c662-42d5-9c75-2301fbc8377a-000000@email.amazonses.com> <01000167f27b4bf1-86cf2251-ed4f-4f6f-88d7-9f2969338a57-000000@email.amazonses.com> In-Reply-To: From: Leif Pedersen Date: Fri, 4 Jan 2019 18:04:27 -0600 Message-ID: Subject: Re: Any plans for EC2 AMIs with RootOnZFS? To: Jay Edwards Cc: Colin Percival , "freebsd-fs@freebsd.org" , Ben Woods , freebsd-cloud@freebsd.org X-Rspamd-Queue-Id: 3B5308AE46 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ofwilsoncreek-com.20150623.gappssmtp.com header.s=20150623 header.b=PSAgNn8b; spf=pass (mx1.freebsd.org: domain of bilbo@hobbiton.org designates 2607:f8b0:4864:20::836 as permitted sender) smtp.mailfrom=bilbo@hobbiton.org X-Spamd-Result: default: False [-3.84 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; URI_COUNT_ODD(1.00)[9]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[ofwilsoncreek-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; FORGED_SENDER(0.30)[leif@ofwilsoncreek.com,bilbo@hobbiton.org]; RCVD_TLS_LAST(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MIME_TRACE(0.00)[0:+,1:+]; FROM_NEQ_ENVFROM(0.00)[leif@ofwilsoncreek.com,bilbo@hobbiton.org]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[ofwilsoncreek-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[ofwilsoncreek.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[6.3.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.65)[ip: (-9.30), ipnet: 2607:f8b0::/32(-2.19), asn: 15169(-1.66), country: US(-0.08)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jan 2019 00:05:06 -0000 I worked out a procedure to convert a UFS filesystem to ZFS, and I've used it successfully many times on AWS and Azure instances. If the disk's data will fit in RAM, it works with a memory disk. Otherwise you can attach a second disk and substitute its device node for $extradisk. It is merely intended as a quick hack to convert after a new installation before putting important data on the machine, so it's not explained or robust. In other words, be sure you understand it before you use it, and very much use it at your own risk. But perhaps one could use the general idea to make a better process. http://devpit.org/wiki/Convert_UFS_to_ZFS Because of `reboot -r`, it works without disturbing the uptime (although it restarts all the services). That's a fun brag if you're into raising eyebrows. ("I changed the boot filesystem without rebooting...can you do that with _your_ favorite OS?") However, I'd suggest rebooting after to test the boot procedure and be sure you don't leave your future self an unpleasant 3 AM surprise. -Leif On Thu, Jan 3, 2019 at 4:38 PM Jay Edwards wrote: > I've got a fairly robust collection of utilities for Google Cloud that: > > 1) Starts up an official FreeBSD image > 2) Mounts a separate drive (the ZFS target) > 3) Installs the release; installs all of the various GCP packages > 4) Configures a bunch of custom stuff applicable to me > 5) Creates an image that is then used as a base for any further custom > images. > > It shouldn't be that difficult to add AWS support. > > Jay Edwards > > > On Thu, Dec 27, 2018 at 8:48 PM Colin Percival > wrote: > > > [Adding freebsd-fs in the hope of finding more ZFS people.] > > > > On 12/23/18 3:51 PM, Colin Percival wrote: > > > On 12/23/18 7:12 AM, Ben Woods wrote: > > >> Is there any plans to provide official FreeBSD AMIs in the AWS > > Marketplace > > >> which use RootOnZFS out of the box? > > > > > > Nothing concrete yet, but it's on my radar. Now that FreeBSD AMIs are > > being > > > published by a separate (release engineering) account, I'm more > > comfortable > > > with working on more "experimental" AMIs. (And if this turns out to be > > widely > > > useful I'll talk to the release engineering team about adding it to the > > set > > > they publish.) > > > > I've created an experimental FreeBSD 12.0 ZFS AMI in the us-east-1 > region: > > ami-0786f5b55d5aa573f > > > > Since I'm basically a ZFS newbie (I run it on my laptop, but only with > the > > default setup from the installer, and I've never used any of the fancy > ZFS > > features) I don't know if I've set everything up properly. So far I have > > one piece of feedback, which is that I should have marked canmount=off > for > > zroot. > > > > Can people please try out this AMI and let me know if there's anything > else > > (ZFS-related) which I should fix? Some time next week I'll fix whatever > > people notice and build AMIs for all the regions. > > > > -- > > Colin Percival > > Security Officer Emeritus, FreeBSD | The power to serve > > Founder, Tarsnap | www.tarsnap.com | Online backups for the truly > paranoid > > _______________________________________________ > > freebsd-cloud@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-cloud > > To unsubscribe, send any mail to "freebsd-cloud-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-cloud@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-cloud > To unsubscribe, send any mail to "freebsd-cloud-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Sat Jan 5 12:40:02 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62898149A143; Sat, 5 Jan 2019 12:40:02 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: from mail-vs1-xe2a.google.com (mail-vs1-xe2a.google.com [IPv6:2607:f8b0:4864:20::e2a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 757EB80D55; Sat, 5 Jan 2019 12:40:01 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: by mail-vs1-xe2a.google.com with SMTP id n13so24233477vsk.4; Sat, 05 Jan 2019 04:40:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=AQ1EbwmDWQV0b+XFN98hU/MksRovIw46xxVuqASzUZY=; b=rMMmbpjGytK0MXcj+ZrGf6W8XGLZB2gj7v/fYENnoJ1H3pqHLdf0xAmpfSzes69kS1 ZRmyTywEm0I2JlBd4r3U724F7i9usCCEku+p6EBAbhP1+cIxUnDmBNDfSA6CNQMSr8Cr 80JmBvNcv+LJrAxJibATk+m35fyuJe5M+lduA0xrcq723Nzs1LO4J/SDWmGGTQ6CqzpA Zeebcvy+gEq5FoBytvkR/8MA6Nz6nSe1KmhM7eNBORBW4vOgLY4v9T5g9PLqPJ5+V9EG o7o5zg18KY9GM1UhKDvCmPFjngihNTXViQ/TCQHMlKuV6B7OCFMz2YtPNhyDopoMJdbV Vs+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=AQ1EbwmDWQV0b+XFN98hU/MksRovIw46xxVuqASzUZY=; b=gO2RkeHXByJ729NH0uItYUNck93JjwcSQanHN5OGyTjOQawy0Hefgpzk8jZBvWNU4r I/Fe9vq/Gk1ygQpqZRnEFaVuucll/O+ECItg40mngu+Ay16nUSOhunua4oO19dV5VO4g kIciBxAqnM6+cj3nLD8/mCp/2/R5olsOGIBGooj0ib05kaS/Y7/kQ4ZZ2sN87NUL94tt c91osT9PfWiD1O3cfbAyrfu/V4fjwhj0vbPWWRMtujRJzyo+QWs+vMbLlmsGdQ+1l/6J UviDF5umHhn+jVtwh/CPhzzG8+bCUehb/6s2Msbl1LO1C3/NyiDtzLfEXSm+labAbTvT aXBA== X-Gm-Message-State: AA+aEWbcOIPN0Ys43uUe2J+LTMaR6C9NfUIiXOmDJkSKu+EN03jqxKP6 u6vHGIQ4c+uBZXaFv2jy1zdgy4871x1w1EAmrio= X-Google-Smtp-Source: AFSGD/U0XWrsWxiFsQVEHUvaS/NjXqPns5rTcbjQFEROy+HqxkVNfZOIJoBwuOVyh7FWu/VtFe3arLUB3aaNsHwfpNY= X-Received: by 2002:a67:1505:: with SMTP id 5mr19585653vsv.20.1546692000537; Sat, 05 Jan 2019 04:40:00 -0800 (PST) MIME-Version: 1.0 References: <01000167dd7b16cd-58ee733f-c662-42d5-9c75-2301fbc8377a-000000@email.amazonses.com> <01000167f27b4c13-41c15731-ae4c-4658-820d-80c4d4fbb01c-000000@email.amazonses.com> In-Reply-To: <01000167f27b4c13-41c15731-ae4c-4658-820d-80c4d4fbb01c-000000@email.amazonses.com> From: Ben Woods Date: Sat, 5 Jan 2019 23:39:57 +1100 Message-ID: Subject: Re: Any plans for EC2 AMIs with RootOnZFS? To: Colin Percival Cc: freebsd-cloud@freebsd.org, "freebsd-fs@freebsd.org" X-Rspamd-Queue-Id: 757EB80D55 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=rMMmbpjG; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of woodsb02@gmail.com designates 2607:f8b0:4864:20::e2a as permitted sender) smtp.mailfrom=woodsb02@gmail.com X-Spamd-Result: default: False [-4.17 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; URI_COUNT_ODD(1.00)[5]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.43)[-0.431,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; SUBJECT_ENDS_QUESTION(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[a.2.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.73)[ip: (-9.71), ipnet: 2607:f8b0::/32(-2.19), asn: 15169(-1.67), country: US(-0.08)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jan 2019 12:40:02 -0000 On Fri, 28 Dec 2018 at 12:43, Colin Percival wrote: > I've created an experimental FreeBSD 12.0 ZFS AMI in the us-east-1 region: > ami-0786f5b55d5aa573f > > Since I'm basically a ZFS newbie (I run it on my laptop, but only with the > default setup from the installer, and I've never used any of the fancy ZFS > features) I don't know if I've set everything up properly. So far I have > one piece of feedback, which is that I should have marked canmount=off for > zroot. > Hi Colin, Thanks a lot for doing this - much appreciated. I have just given it a try, and have the following feedback for you: 1. ZFS DATASETS AND PROPERTIES SHOULD MATCH INSTALLER DEFAULT To avoid any confusion or unexpected behaviour, I believe the list of ZFS datasets and their properties should exactly match the bsdinstall defaults (especially for releases). zpool create: https://svnweb.freebsd.org/base/head/usr.sbin/bsdinstall/scripts/zfsboot?revision=342283&view=markup#l1308 zfs datasets and properties: https://svnweb.freebsd.org/base/head/usr.sbin/bsdinstall/scripts/zfsboot?revision=342283&view=markup#l141 Looking at how you created the zpool and zfs datasets (using the "zpool history zroot" command), the following changes would achieve this outcome: --- zfs-commands.old +++ zfs-commands.new @@ -1,19 +1,15 @@ -zpool create -o altroot=/mnt zroot ada0p2 -zfs set compress=on zroot +zpool create -o altroot=/mnt -O compress=lz4 -O atime=off -m none -f zroot ada0p2 zfs create -o mountpoint=none zroot/ROOT zfs create -o mountpoint=/ -o canmount=noauto zroot/ROOT/default zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp -zfs create -o canmount=off -o mountpoint=/usr zroot/usr +zfs create -o mountpoint=/usr -o canmount=off zroot/usr zfs create zroot/usr/home -zfs create -o exec=off -o setuid=off zroot/usr/src -zfs create zroot/usr/obj -zfs create -o mountpoint=/usr/ports -o setuid=off zroot/usr/ports -zfs create -o exec=off -o setuid=off zroot/usr/ports/distfiles -zfs create -o exec=off -o setuid=off zroot/usr/ports/packages -zfs create -o canmount=off -o mountpoint=/var zroot/var +zfs create -o setuid=off zroot/usr/ports +zfs create zroot/usr/src +zfs create -o mountpoint=/var -o canmount=off zroot/var zfs create -o exec=off -o setuid=off zroot/var/audit zfs create -o exec=off -o setuid=off zroot/var/crash zfs create -o exec=off -o setuid=off zroot/var/log -zfs create -o atime=on -o exec=off -o setuid=off zroot/var/mail -zfs create -o exec=on -o setuid=off zroot/var/tmp +zfs create -o atime=on zroot/var/mail +zfs create -o setuid=off zroot/var/tmp zpool set bootfs=zroot/ROOT/default zroot 2. LOADER.CONF SETTINGS The following settings should be included in /boot/loader.conf: kern.geom.label.disk_ident.enable="0" kern.geom.label.gptid.enable="0" 3. SYSCTL.CONF SETTINGS The following settings should be included in /etc/sysctl.conf: vfs.zfs.min_auto_ashift=12 4. THE GPT TABLE IS INITIALLY CORRUPT I suspect this is because I used a 50Gb disk, and it hasn't been expanded correctly. The following error was in DMESG and I suspect is related: GEOM: ada0: the secondary GPT header is not in the last LBA. I was able to fix this with the following commands on firstboot: # gpart recover ada0 # gpart resize -i 2 ada0 # zpool online -e zroot ada0p2 The full background of my investigation and resolution is shown below: # gpart show => 40 20971440 ada0 GPT (50G) [CORRUPT] 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 20967424 2 freebsd-zfs (10G) 20969472 2008 - free - (1.0M) # gpart recover ada0 ada0 recovered # gpart show => 40 104857520 ada0 GPT (50G) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 20967424 2 freebsd-zfs (10G) 20969472 83888088 - free - (40G) # gpart resize -i 2 ada0 ada0p2 resized # gpart show => 40 104857520 ada0 GPT (50G) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 104855512 2 freebsd-zfs (50G) # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 9.50G 1.25G 8.25G - - 0% 13% 1.00x ONLINE - # zpool online -e zroot ada0p2 # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 49.5G 1.25G 48.3G - - 0% 2% 1.00x ONLINE - Thanks again for your efforts. Regards, Ben -- From: Benjamin Woods woodsb02@gmail.com From owner-freebsd-fs@freebsd.org Sat Jan 5 22:14:28 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4248D1424E71; Sat, 5 Jan 2019 22:14:28 +0000 (UTC) (envelope-from SRS0=/O5A=PN=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A54F06F84A; Sat, 5 Jan 2019 22:14:27 +0000 (UTC) (envelope-from SRS0=/O5A=PN=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 2553528417; Sat, 5 Jan 2019 23:14:16 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 4BDAD28416; Sat, 5 Jan 2019 23:14:14 +0100 (CET) Subject: Re: Any plans for EC2 AMIs with RootOnZFS? To: Ben Woods , Colin Percival Cc: "freebsd-fs@freebsd.org" , freebsd-cloud@freebsd.org References: <01000167dd7b16cd-58ee733f-c662-42d5-9c75-2301fbc8377a-000000@email.amazonses.com> <01000167f27b4c13-41c15731-ae4c-4658-820d-80c4d4fbb01c-000000@email.amazonses.com> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: Date: Sat, 5 Jan 2019 23:14:13 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: A54F06F84A X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [5.56 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: elsa.codelab.cz]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=/O5A=PN=quip.cz=000.fbsd@elsa.codelab.cz]; FREEMAIL_TO(0.00)[gmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[209.16.49.86.zen.spamhaus.org : 127.0.0.11]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=/O5A=PN=quip.cz=000.fbsd@elsa.codelab.cz]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_SPAM_SHORT(0.95)[0.947,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[quip.cz]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.97)[0.974,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(1.00)[0.997,0]; RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0]; IP_SCORE(0.45)[ip: (1.18), ipnet: 94.124.104.0/21(0.59), asn: 42000(0.47), country: CZ(0.04)]; R_SPF_NA(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jan 2019 22:14:28 -0000 Ben Woods wrote on 2019/01/05 13:39: > On Fri, 28 Dec 2018 at 12:43, Colin Percival wrote: > >> I've created an experimental FreeBSD 12.0 ZFS AMI in the us-east-1 region: >> ami-0786f5b55d5aa573f >> >> Since I'm basically a ZFS newbie (I run it on my laptop, but only with the >> default setup from the installer, and I've never used any of the fancy ZFS >> features) I don't know if I've set everything up properly. So far I have >> one piece of feedback, which is that I should have marked canmount=off for >> zroot. >> > Hi Colin, > > Thanks a lot for doing this - much appreciated. > > I have just given it a try, and have the following feedback for you: > > 1. ZFS DATASETS AND PROPERTIES SHOULD MATCH INSTALLER DEFAULT > To avoid any confusion or unexpected behaviour, I believe the list of ZFS > datasets and their properties should exactly match the bsdinstall defaults > (especially for releases). > zpool create: > https://svnweb.freebsd.org/base/head/usr.sbin/bsdinstall/scripts/zfsboot?revision=342283&view=markup#l1308 > zfs datasets and properties: > https://svnweb.freebsd.org/base/head/usr.sbin/bsdinstall/scripts/zfsboot?revision=342283&view=markup#l141 > > Looking at how you created the zpool and zfs datasets (using the "zpool > history zroot" command), the following changes would achieve this outcome: > --- zfs-commands.old > +++ zfs-commands.new > @@ -1,19 +1,15 @@ > -zpool create -o altroot=/mnt zroot ada0p2 > -zfs set compress=on zroot > +zpool create -o altroot=/mnt -O compress=lz4 -O atime=off -m none -f zroot > ada0p2 > zfs create -o mountpoint=none zroot/ROOT > zfs create -o mountpoint=/ -o canmount=noauto zroot/ROOT/default > zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp > -zfs create -o canmount=off -o mountpoint=/usr zroot/usr > +zfs create -o mountpoint=/usr -o canmount=off zroot/usr > zfs create zroot/usr/home > -zfs create -o exec=off -o setuid=off zroot/usr/src > -zfs create zroot/usr/obj > -zfs create -o mountpoint=/usr/ports -o setuid=off zroot/usr/ports > -zfs create -o exec=off -o setuid=off zroot/usr/ports/distfiles > -zfs create -o exec=off -o setuid=off zroot/usr/ports/packages > -zfs create -o canmount=off -o mountpoint=/var zroot/var > +zfs create -o setuid=off zroot/usr/ports > +zfs create zroot/usr/src > +zfs create -o mountpoint=/var -o canmount=off zroot/var > zfs create -o exec=off -o setuid=off zroot/var/audit > zfs create -o exec=off -o setuid=off zroot/var/crash > zfs create -o exec=off -o setuid=off zroot/var/log > -zfs create -o atime=on -o exec=off -o setuid=off zroot/var/mail > -zfs create -o exec=on -o setuid=off zroot/var/tmp > +zfs create -o atime=on zroot/var/mail > +zfs create -o setuid=off zroot/var/tmp > zpool set bootfs=zroot/ROOT/default zroot I think the opposite - bsdinstall shoul be fixed to use exec=off and setuid=off. Just my $0.02 Miroslav Lachman From owner-freebsd-fs@freebsd.org Sat Jan 5 22:31:36 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 122EB1425602; Sat, 5 Jan 2019 22:31:36 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: from mail-vs1-xe36.google.com (mail-vs1-xe36.google.com [IPv6:2607:f8b0:4864:20::e36]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3D2E5700D0; Sat, 5 Jan 2019 22:31:35 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: by mail-vs1-xe36.google.com with SMTP id z23so21465579vsj.11; Sat, 05 Jan 2019 14:31:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WOzm8TduS/naFb1n99u6o6s28Jilqgg8TF0DvE35hdY=; b=Bx6lsCrFMyDIEYS244UyzaQdc+EMTo04gFUTwJFXZVTJo9Q6NbnE7U1jPZsAw9Su9i nDANNB24Fu3ZJQUyTj641Gh6yyBj+gfS7eLOW5IdZgSEbQz++nWpN8VYJPpCq/O9ZjDp +pr6mJGxaVaikFheGBpPYiq+n1YWYaF6Hku4c3Md/SFqnyop1WYhZwEotcRqxR1LzuUi kk7rU9hfOZEZ7xQZkurCd0BbGX1AcVrOUii0vADn1DRgGeHglKZSZY+rlAgbk37fr0m4 4fQqxOn+dA1l3VHArIWK51R5dV0suvNOvOLx3NQZM+pmyuvDOCgTa346KL0UGrvnC2w8 HDoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WOzm8TduS/naFb1n99u6o6s28Jilqgg8TF0DvE35hdY=; b=YQndWkggwa4IX3a9L2e+8Nff5Z7n9SMKjd7KE/Ct+CSiQJhiazXUt0esm9e9RNV5cw oX9m3FoBXNARSKQgvcXrOLXkzQdJFCTA+wQ6NRreSb/jcp57JMcPrCA6OOExTpb2fVRQ SlrPuj1UUe9gp+MeZfm3BlastRo5/heYL/wk4rHDxgjNnukfSypyh/UXuHQdIH8p7k2b bPfIEMdFbgQjlyAWQkjo4MS+Zj2vOAg3XCgc3Nq4dxwB+isXHGowoopfZiRgyusNcmxW 4y5a6AjXrsLaD7QtyMQTDz+QGwn8MT01ilnMOgpvTdpH9ZAC4Y5hW294OOkTfQPsI3s1 G4eQ== X-Gm-Message-State: AA+aEWbLGmeE+hsLA+25FpFTRBNMOAFV48+HIf8v8+nanfINDfUVGaTN UkhqXeS+ZYBiAD8pcf5mQNUvajL0QBaNsc+r7TDwXQ== X-Google-Smtp-Source: AFSGD/UKGqlQigAJQms3t2Q3IRdul/3Dj059cG88r+M9Zm6frLzbw1OFW2VqGGg6nRDS1uW2LIadgv8Hu0UdRjwLzss= X-Received: by 2002:a67:1505:: with SMTP id 5mr20097365vsv.20.1546727494514; Sat, 05 Jan 2019 14:31:34 -0800 (PST) MIME-Version: 1.0 References: <01000167dd7b16cd-58ee733f-c662-42d5-9c75-2301fbc8377a-000000@email.amazonses.com> <01000167f27b4c13-41c15731-ae4c-4658-820d-80c4d4fbb01c-000000@email.amazonses.com> In-Reply-To: From: Ben Woods Date: Sun, 6 Jan 2019 09:31:23 +1100 Message-ID: Subject: Re: Any plans for EC2 AMIs with RootOnZFS? To: Miroslav Lachman <000.fbsd@quip.cz> Cc: Colin Percival , freebsd-cloud@freebsd.org, "freebsd-fs@freebsd.org" X-Rspamd-Queue-Id: 3D2E5700D0 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Bx6lsCrF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of woodsb02@gmail.com designates 2607:f8b0:4864:20::e36 as permitted sender) smtp.mailfrom=woodsb02@gmail.com X-Spamd-Result: default: False [-5.73 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.97)[-0.974,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; SUBJECT_ENDS_QUESTION(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[6.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.75)[ip: (-9.80), ipnet: 2607:f8b0::/32(-2.20), asn: 15169(-1.67), country: US(-0.08)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jan 2019 22:31:36 -0000 On Sun, 6 Jan 2019 at 9:14 am, Miroslav Lachman <000.fbsd@quip.cz> wrote: > I think the opposite - bsdinstall shoul be fixed to use exec=3Doff and > setuid=3Doff. > > Just my $0.02 > > Miroslav Lachman > Hi Miroslav, I agree with you there. I just think that until that happens, the FreeBSD AWS EC2 images shouldn=E2= =80=99t take it upon themself to change the defaults - the place to change it is in bsdinstall for the next release. Regards, Ben --=20 -- From: Benjamin Woods woodsb02@gmail.com From owner-freebsd-fs@freebsd.org Sat Jan 5 23:54:05 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EEED314292EB for ; Sat, 5 Jan 2019 23:54:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8713E74632 for ; Sat, 5 Jan 2019 23:54:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4AA1414292EA; Sat, 5 Jan 2019 23:54:04 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 393EC14292E9 for ; Sat, 5 Jan 2019 23:54:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CBE6F7462C for ; Sat, 5 Jan 2019 23:54:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 1CB12DE0 for ; Sat, 5 Jan 2019 23:54:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x05Ns22J011958 for ; Sat, 5 Jan 2019 23:54:02 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x05Ns2uR011957 for fs@FreeBSD.org; Sat, 5 Jan 2019 23:54:02 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 229614] ZFS lockup in zil_commit_impl Date: Sat, 05 Jan 2019 23:53:56 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: nivedita@alum.mit.edu X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: allanjude@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jan 2019 23:54:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229614 Arvind changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nivedita@alum.mit.edu --- Comment #46 from Arvind --- Coming here by way of https://github.com/zfsonlinux/zfs/issues/7425 Re ZoL comment #43, the unlinked check was removed in https://github.com/zfsonlinux/zfs/commit/987014903f9d36783547188b6ad00f01d9= a076bd The check apparently caused some issues with xattr functionality, not sure = that applies in FreeBSD but this loop is still happening in ZoL potentially looks like. Should we just try limiting the number of retries in there? --=20 You are receiving this mail because: You are on the CC list for the bug.=