From owner-freebsd-fs@freebsd.org Sun Feb 19 11:13:08 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CC7FCE53D7 for ; Sun, 19 Feb 2017 11:13:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0C6D2A94 for ; Sun, 19 Feb 2017 11:13:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1JBD7Xj092876 for ; Sun, 19 Feb 2017 11:13:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Sun, 19 Feb 2017 11:13:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 11:13:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #10 from Lev A. Serebryakov --- (In reply to Andriy Gapon from comment #9) Ok, I've replaced L2ARC to whole Samsung 850 EVO 250 SSD on "mpr" (LSI-3008) HBA and rebuilt system from r313940 (stable/11) + this patch. Lets see! --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sun Feb 19 19:51:00 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ACC81CE502B for ; Sun, 19 Feb 2017 19:51:00 +0000 (UTC) (envelope-from "") Received: from cdptpa-oedge-vip.email.rr.com (cdptpa-outbound-snat.email.rr.com [107.14.166.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cdptpa-oedge", Issuer "cdptpa-oedge" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 776A41F58 for ; Sun, 19 Feb 2017 19:51:00 +0000 (UTC) (envelope-from "") Received: from [107.14.174.246] ([107.14.174.246:25913] helo=cdptpa-fep02.email.rr.com) by cdptpa-omsmta02 (envelope-from <>) (ecelerity 3.6.9.48312 r(Core:3.6.9.0)) with ESMTP id EC/B2-16480-D17F9A85; Sun, 19 Feb 2017 19:50:53 +0000 Subject: Re: You have notifications pending To: freebsd-fs@freebsd.org From: "Auto-reply from llescoe@ca.rr.com" In-Reply-To: Precedence: bulk Date: Sun, 19 Feb 2017 19:50:53 +0000 Message-ID: <20170219195053.DKNW5400.cdptpa-fep02.email.rr.com@cdptpa-fep02> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-RR-Connecting-IP: 107.14.168.7:25 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 19:51:00 -0000 Notice: I only check my email twice a week once on the weekends and once during the week. If your message requires an immediate response please phone me so that I may be responsive to your need in a more timely fashion. I sincerely appreciate it. From owner-freebsd-fs@freebsd.org Sun Feb 19 20:51:43 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54902CE539D for ; Sun, 19 Feb 2017 20:51:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 43FF89D2 for ; Sun, 19 Feb 2017 20:51:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1JKpgnT040007 for ; Sun, 19 Feb 2017 20:51:43 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 217062] for file systems mounted with -o noexec, exec=off property does not work for mmap Date: Sun, 19 Feb 2017 20:51:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 20:51:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D217062 --- Comment #7 from commit-hook@freebsd.org --- A commit references this bug: Author: kib Date: Sun Feb 19 20:51:05 UTC 2017 New revision: 313967 URL: https://svnweb.freebsd.org/changeset/base/313967 Log: Apply noexec mount option for mmap(PROT_EXEC). Right now the noexec mount option disallows image activators to try execve the files on the mount point. Also, after r127187, noexec also limits max_prot map entries permissions for mappings of files from such mounts, but not the actual mapping permissions. As result, the API behaviour is inconsistent. The files from noexec mount can be mapped with PROT_EXEC, but if mprotect(2) drops execution permission, it cannot be re-enabled later. Make this consistent logically and aligned with behaviour of other systems, by disallowing PROT_EXEC for mmap(2). Note that this change only ensures aligned results from mmap(2) and mprotect(2), it does not prevent actual code execution from files coming from noexec mount. Such files can always be read into anonymous executable memory and executed from there. Reported by: shamaz.mazum@gmail.com PR: 217062 Reviewed by: alc Sponsored by: The FreeBSD Foundation MFC after: 1 week Changes: head/sys/fs/devfs/devfs_vnops.c head/sys/kern/vfs_vnops.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sun Feb 19 21:00:03 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7415CE544B for ; Sun, 19 Feb 2017 21:00:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9E0B7B29 for ; Sun, 19 Feb 2017 21:00:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1JL01f6054691 for ; Sun, 19 Feb 2017 21:00:03 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201702192100.v1JL01f6054691@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention Date: Sun, 19 Feb 2017 21:00:03 +0000 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 21:00:03 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 140068 | [smbfs] [patch] smbfs does not allow semicolon in Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 203419 | solaris assert: (dn->dn_phys->dn_nlevels == 0 && Open | 211491 | System hangs after "Uptime" on reboot with ZFS 7 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Sun Feb 19 23:15:02 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60EA0CE68D6 for ; Sun, 19 Feb 2017 23:15:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5000BBDB for ; Sun, 19 Feb 2017 23:15:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1JNF1pW009122 for ; Sun, 19 Feb 2017 23:15:02 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 217062] for file systems mounted with -o noexec, exec=off property does not work for mmap Date: Sun, 19 Feb 2017 23:15:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: op@hardenedbsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: flagtypes.name cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 23:15:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D217062 op@hardenedbsd.org changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |mfc-stable10?, | |mfc-stable11? CC| |op@hardenedbsd.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Feb 20 10:28:33 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D9BECCE55A9; Mon, 20 Feb 2017 10:28:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C52691951; Mon, 20 Feb 2017 10:28:32 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA19514; Mon, 20 Feb 2017 12:28:24 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1cflCi-000L58-E8; Mon, 20 Feb 2017 12:28:24 +0200 Subject: Re: major code change for .zfs To: FreeBSD Current , freebsd-fs , freebsd-stable List References: <34c1e635-4f38-d483-bcf9-768311d7c726@FreeBSD.org> From: Andriy Gapon Message-ID: Date: Mon, 20 Feb 2017 12:27:27 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: <34c1e635-4f38-d483-bcf9-768311d7c726@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Feb 2017 10:28:34 -0000 On 23/08/2016 11:43, Andriy Gapon wrote: > > Please review and test a change to .zfs code that is intended to make the code > aligned with FreeBSD VFS and, as such, more stable: > https://reviews.freebsd.org/D7421 > > The change removes two features. > .zfs/shares is gone because it was unused on FreeBSD anyway. We can restore > that when we need it. > An ability to take a snapshot by creating a directory under .zfs/snapshot is > removed. I hope that you didn't use it. Please do not start using it now :-) > Again, this feature can be restored with some work. > The reason I removed it is that its companion features of destroying and > renaming snapshots were already missing on FreeBSD, and properly implementing > the feature required some more work. This is a heads-up that I am going to commit the change. If you have objections or concerns please speak up. Thanks! -- Andriy Gapon From owner-freebsd-fs@freebsd.org Mon Feb 20 11:12:10 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 490EECE64ED for ; Mon, 20 Feb 2017 11:12:10 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wr0-x22e.google.com (mail-wr0-x22e.google.com [IPv6:2a00:1450:400c:c0c::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E9AE31562 for ; Mon, 20 Feb 2017 11:12:09 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wr0-x22e.google.com with SMTP id 89so57342862wrr.3 for ; Mon, 20 Feb 2017 03:12:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=+TMedQWtZQWgYzp2g/J2pQg4hL4OHv7Fg6sA+GCXL3M=; b=sYjHzWoRy5GFBglnb9t8o9whauLfklLqGzAjg402pqwMYRE3kniZJDCnRjirA/+CNN 5aq6HMopBewQw9FZf31SL9loz60xLP3/XOagy797Dzm9t3SBF0Ydky3oNcwZQgZ+uj9j M6AxaUWLCazFnC9NONzkfqQqkl5Kcn3QghbhhYbI6DugsMPKfYU8NdgPmg6GuBVk+DWW pDln38fERJGuDjsCUXW/EPvst7h5iimowOvz5D2W7tB/6RcrSzEgzqgb61RabPi138C7 V93y1pijaq72eoYwPR4w896HVHwe//Kde8iWLjo+ifY4rvC77mRo+puJAltGSI4KzmJ1 yf/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=+TMedQWtZQWgYzp2g/J2pQg4hL4OHv7Fg6sA+GCXL3M=; b=TdHm3gmOBNjg3FKTGss8D/hjqisjYH9N2tSsStMtNs7YDgQZv1UOxYVtArUB5pg6JK RSqvXVXmSYTAJUiDm2/dYqzqMBIQZ8ir3rd76VcQ4gQaOU6pDUKBISj2fSm0UxDtVyyG swAN0+7SZgHQpD9+1T69RTr+VRCYr1pWMrNn5+LduXOLy/xmLAmGdb4EVOcKu8eVZRxz OequUKhtTlqDb6iDjn3vWM3AuYS9CE/1NLzJN2Pgaq04UmbTukGBJtZqpy50JVANdLH5 Vh4t4Z2z5Ji6zloHKGFq82WwnFro367nreU8i4R0InE7nuvw96E71nUMFllY9sRIUNMA oeiQ== X-Gm-Message-State: AMke39mIpxVS7AX1beimJ/t7uW4WhWqrQxS78yA/68jbgJVvqdnAvxHEYkErdryBuaYI9h9f X-Received: by 10.223.140.18 with SMTP id z18mr15411587wra.6.1487589128034; Mon, 20 Feb 2017 03:12:08 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id d42sm24329486wrd.7.2017.02.20.03.12.06 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Feb 2017 03:12:07 -0800 (PST) Subject: Re: major code change for .zfs To: freebsd-fs@freebsd.org References: <34c1e635-4f38-d483-bcf9-768311d7c726@FreeBSD.org> From: Steven Hartland Message-ID: Date: Mon, 20 Feb 2017 11:12:06 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Feb 2017 11:12:10 -0000 On 20/02/2017 10:27, Andriy Gapon wrote: > On 23/08/2016 11:43, Andriy Gapon wrote: >> Please review and test a change to .zfs code that is intended to make the code >> aligned with FreeBSD VFS and, as such, more stable: >> https://reviews.freebsd.org/D7421 >> >> The change removes two features. >> .zfs/shares is gone because it was unused on FreeBSD anyway. We can restore >> that when we need it. >> An ability to take a snapshot by creating a directory under .zfs/snapshot is >> removed. I hope that you didn't use it. Please do not start using it now :-) >> Again, this feature can be restored with some work. >> The reason I removed it is that its companion features of destroying and >> renaming snapshots were already missing on FreeBSD, and properly implementing >> the feature required some more work. > This is a heads-up that I am going to commit the change. > If you have objections or concerns please speak up. > Thanks! > The only thing I would suggest is that given its invasive nature you way want to add an entry to UPDATING? Regard Steve From owner-freebsd-fs@freebsd.org Mon Feb 20 11:45:44 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29F2ACE6FB0 for ; Mon, 20 Feb 2017 11:45:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1433C13D7 for ; Mon, 20 Feb 2017 11:45:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1KBjhcA066752 for ; Mon, 20 Feb 2017 11:45:43 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Mon, 20 Feb 2017 11:45:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Feb 2017 11:45:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #11 from Lev A. Serebryakov --- Same result. 11.0-STABLE FreeBSD 11.0-STABLE #8 r313940M: Sun Feb 19 15:16:42 MSK 2017 Attached patch was applied! % sudo smartctl -A /dev/da5 smartctl 6.5 2016-05-07 r4318 [FreeBSD 11.0-STABLE amd64] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org =3D=3D=3D START OF INFORMATION SECTION =3D=3D=3D Model Family: Samsung based SSDs Device Model: Samsung SSD 850 EVO 250GB Serial Number: S2R4NB0J115081X LU WWN Device Id: 5 002538 d41a03ed9 Firmware Version: EMT02B6Q User Capacity: 250,059,350,016 bytes [250 GB] Sector Size: 512 bytes logical/physical Rotation Rate: Solid State Device Form Factor: 2.5 inches Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-2, ATA8-ACS T13/1699-D revision 4c SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Mon Feb 20 14:41:13 2017 MSK SMART support is: Available - device has SMART capability. SMART support is: Enabled ... % zpool list -v NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALT= ROOT zroot 95.5G 18.1G 77.4G - 16% 18% 1.00x ONLINE - gpt/root 95.5G 18.1G 77.4G - 16% 18% zstor 13.6T 9.15T 4.48T - 25% 67% 1.00x ONLINE - raidz1 13.6T 9.15T 4.48T - 25% 67% da1 - - - - - - da0 - - - - - - da2 - - - - - - da3 - - - - - - da4 - - - - - - cache - - - - - - da5 233G 526G 16.0E - 0% 225% % sysctl -a | grep l2 kern.features.linuxulator_v4l2: 1 kern.cam.ctl2cam.max_sense: 252 vfs.zfs.l2c_only_size: 0 vfs.zfs.l2arc_norw: 1 vfs.zfs.l2arc_feed_again: 1 vfs.zfs.l2arc_noprefetch: 1 vfs.zfs.l2arc_feed_min_ms: 200 vfs.zfs.l2arc_feed_secs: 1 vfs.zfs.l2arc_headroom: 2 vfs.zfs.l2arc_write_boost: 8388608 vfs.zfs.l2arc_write_max: 8388608 vfs.cache.numfullpathfail2: 0 kstat.zfs.misc.arcstats.l2_write_buffer_list_null_iter: 4522 kstat.zfs.misc.arcstats.l2_write_buffer_list_iter: 567486 kstat.zfs.misc.arcstats.l2_write_buffer_bytes_scanned: 10356335975424 kstat.zfs.misc.arcstats.l2_write_pios: 131233 kstat.zfs.misc.arcstats.l2_write_buffer_iter: 142392 kstat.zfs.misc.arcstats.l2_write_full: 34822 kstat.zfs.misc.arcstats.l2_write_not_cacheable: 6310602 kstat.zfs.misc.arcstats.l2_write_io_in_progress: 376 kstat.zfs.misc.arcstats.l2_write_in_l2: 63312026 kstat.zfs.misc.arcstats.l2_write_spa_mismatch: 3663622 kstat.zfs.misc.arcstats.l2_write_passed_headroom: 255811 kstat.zfs.misc.arcstats.l2_write_trylock_fail: 6901 kstat.zfs.misc.arcstats.l2_padding_needed: 0 kstat.zfs.misc.arcstats.l2_hdr_size: 93854992 kstat.zfs.misc.arcstats.l2_asize: 565424867840 kstat.zfs.misc.arcstats.l2_size: 567082335232 kstat.zfs.misc.arcstats.l2_io_error: 0 kstat.zfs.misc.arcstats.l2_cksum_bad: 430864 kstat.zfs.misc.arcstats.l2_abort_lowmem: 3 kstat.zfs.misc.arcstats.l2_free_on_write: 273 kstat.zfs.misc.arcstats.l2_evict_l1cached: 69699 kstat.zfs.misc.arcstats.l2_evict_reading: 0 kstat.zfs.misc.arcstats.l2_evict_lock_retry: 10 kstat.zfs.misc.arcstats.l2_writes_lock_retry: 4 kstat.zfs.misc.arcstats.l2_writes_error: 0 kstat.zfs.misc.arcstats.l2_writes_done: 131233 kstat.zfs.misc.arcstats.l2_writes_sent: 131233 kstat.zfs.misc.arcstats.l2_write_bytes: 648248758272 kstat.zfs.misc.arcstats.l2_read_bytes: 326535054336 kstat.zfs.misc.arcstats.l2_rw_clash: 0 kstat.zfs.misc.arcstats.l2_feeds: 142392 kstat.zfs.misc.arcstats.l2_misses: 3610592 kstat.zfs.misc.arcstats.l2_hits: 1151244 kstat.zfs.misc.arcstats.evict_l2_skip: 0 kstat.zfs.misc.arcstats.evict_l2_ineligible: 351227303936 kstat.zfs.misc.arcstats.evict_l2_eligible: 21983154688 kstat.zfs.misc.arcstats.evict_l2_cached: 1323198704640 % kstat.zfs.misc.arcstats.l2_cksum_bad starts to rise right after L2ARC "overfill"! --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Feb 20 15:37:24 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 642E5CE5C28 for ; Mon, 20 Feb 2017 15:37:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 535191AC0 for ; Mon, 20 Feb 2017 15:37:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1KFbOIY071861 for ; Mon, 20 Feb 2017 15:37:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Mon, 20 Feb 2017 15:37:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: remi.guyomarch@ign.fr X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Feb 2017 15:37:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 R=C3=A9mi Guyomarch changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |remi.guyomarch@ign.fr --- Comment #12 from R=C3=A9mi Guyomarch --- Same thing here, running 10.3-STABLE r313140. It did NOT happen on r301989. This is a large virtual NAS, offering both NFSv3 and SMB shares. Cache devi= ces are also virtualized, TRIM isn't running here. # zpool list -v tank NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALT= ROOT tank 288T 174T 114T - 19% 60% 1.00x ONLINE - raidz2 48,0T 28,9T 19,0T - 19% 60% da9 - - - - - - da10 - - - - - - da11 - - - - - - da12 - - - - - - da13 - - - - - - da14 - - - - - - raidz2 48,0T 28,9T 19,0T - 19% 60% da15 - - - - - - da16 - - - - - - da17 - - - - - - da18 - - - - - - da19 - - - - - - da20 - - - - - - raidz2 48,0T 28,9T 19,0T - 19% 60% da21 - - - - - - da22 - - - - - - da23 - - - - - - da24 - - - - - - da25 - - - - - - da26 - - - - - - raidz2 48,0T 29,0T 19,0T - 19% 60% da27 - - - - - - da28 - - - - - - da29 - - - - - - da30 - - - - - - da31 - - - - - - da32 - - - - - - raidz2 48,0T 28,9T 19,0T - 19% 60% da33 - - - - - - da34 - - - - - - da35 - - - - - - da36 - - - - - - da37 - - - - - - da38 - - - - - - raidz2 48,0T 28,9T 19,0T - 19% 60% da39 - - - - - - da40 - - - - - - da41 - - - - - - da42 - - - - - - da43 - - - - - - da44 - - - - - - log - - - - - - mirror 2,98G 2,01M 2,98G - 20% 0% da1 - - - - - - da2 - - - - - - cache - - - - - - da3 256G 764G 16,0E - 0% 298% da4 256G 757G 16,0E - 0% 295% da5 256G 762G 16,0E - 0% 297% da6 256G 747G 16,0E - 0% 291% da7 256G 776G 16,0E - 0% 303% da8 256G 743G 16,0E - 0% 290% # zfs-stats -a ------------------------------------------------------------------------ ZFS Subsystem Report Mon Feb 20 16:33:13 2017 ------------------------------------------------------------------------ System Information: Kernel Version: 1003511 (osreldate) Hardware Platform: amd64 Processor Architecture: amd64 ZFS Storage pool Version: 5000 ZFS Filesystem Version: 5 FreeBSD 10.3-STABLE #2 r313140M: Fri Feb 3 09:38:12 CET 2017 root 16:33 up 7 days, 8:32, 1 user, load averages: 0,16 0,37 0,55 ------------------------------------------------------------------------ System Memory: 0.00% 5.28 MiB Active, 0.47% 670.72 MiB Inact 89.61% 125.70 GiB Wired, 0.00% 0 Cache 9.92% 13.92 GiB Free, 0.00% 4.00 KiB Gap Real Installed: 160.00 GiB Real Available: 89.98% 143.97 GiB Real Managed: 97.44% 140.28 GiB Logical Total: 160.00 GiB Logical Used: 90.89% 145.42 GiB Logical Free: 9.11% 14.58 GiB Kernel Memory: 1.30 GiB Data: 97.94% 1.27 GiB Text: 2.06% 27.29 MiB Kernel Memory Map: 140.28 GiB Size: 82.86% 116.24 GiB Free: 17.14% 24.04 GiB ------------------------------------------------------------------------ ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 23.76m Recycle Misses: 0 Mutex Misses: 36.19k Evict Skips: 6.43k ARC Size: 83.09% 115.73 GiB Target Size: (Adaptive) 83.11% 115.76 GiB Min Size (Hard Limit): 12.50% 17.41 GiB Max Size (High Water): 8:1 139.28 GiB ARC Size Breakdown: Recently Used Cache Size: 62.61% 72.48 GiB Frequently Used Cache Size: 37.39% 43.28 GiB ARC Hash Breakdown: Elements Max: 14.41m Elements Current: 98.47% 14.19m Collisions: 16.02m Chain Max: 7 Chains: 2.27m ------------------------------------------------------------------------ ARC Efficiency: 3.28b Cache Hit Ratio: 18.94% 620.69m Cache Miss Ratio: 81.06% 2.66b Actual Hit Ratio: 5.26% 172.47m Data Demand Efficiency: 30.02% 138.53m Data Prefetch Efficiency: 82.77% 124.81m CACHE HITS BY CACHE LIST: Anonymously Used: 71.34% 442.81m Most Recently Used: 2.11% 13.07m Most Frequently Used: 25.68% 159.41m Most Recently Used Ghost: 0.02% 102.06k Most Frequently Used Ghost: 0.86% 5.31m CACHE HITS BY DATA TYPE: Demand Data: 6.70% 41.58m Prefetch Data: 16.64% 103.30m Demand Metadata: 3.92% 24.33m Prefetch Metadata: 72.74% 451.49m CACHE MISSES BY DATA TYPE: Demand Data: 3.65% 96.95m Prefetch Data: 0.81% 21.51m Demand Metadata: 95.51% 2.54b Prefetch Metadata: 0.03% 880.35k ------------------------------------------------------------------------ L2 ARC Summary: (DEGRADED) Passed Headroom: 975.75k Tried Lock Failures: 121.14m IO In Progress: 5 Low Memory Aborts: 217 Free on Write: 181.66k Writes While Full: 46.56k R/W Clashes: 0 Bad Checksums: 3.00m IO Errors: 0 SPA Mismatch: 1.97b L2 ARC Size: (Adaptive) 6.82 TiB Header Size: 0.01% 973.67 MiB L2 ARC Evicts: Lock Retries: 120 Upon Reading: 0 L2 ARC Breakdown: 2.66b Hit Ratio: 0.61% 16.16m Miss Ratio: 99.39% 2.64b Feeds: 688.84k L2 ARC Buffer: Bytes Scanned: 5.87 PiB Buffer Iterations: 688.84k List Iterations: 2.75m NULL List Iterations: 2.97k L2 ARC Writes: Writes Sent: 100.00% 365.54k ------------------------------------------------------------------------ File-Level Prefetch: (HEALTHY) DMU Efficiency: 16.05b Hit Ratio: 2.21% 354.01m Miss Ratio: 97.79% 15.69b Colinear: 0 Hit Ratio: 100.00% 0 Miss Ratio: 100.00% 0 Stride: 0 Hit Ratio: 100.00% 0 Miss Ratio: 100.00% 0 DMU Misc: Reclaim: 0 Successes: 100.00% 0 Failures: 100.00% 0 Streams: 0 +Resets: 100.00% 0 -Resets: 100.00% 0 Bogus: 0 ------------------------------------------------------------------------ VDEV Cache Summary: 5.56m Hit Ratio: 22.19% 1.23m Miss Ratio: 65.26% 3.63m Delegations: 12.55% 696.99k ------------------------------------------------------------------------ ZFS Tunables (sysctl): kern.maxusers 9550 vm.kmem_size 150625865728 vm.kmem_size_scale 1 vm.kmem_size_min 0 vm.kmem_size_max 1319413950874 vfs.zfs.trim.max_interval 1 vfs.zfs.trim.timeout 30 vfs.zfs.trim.txg_delay 32 vfs.zfs.trim.enabled 0 vfs.zfs.vol.unmap_enabled 1 vfs.zfs.vol.mode 1 vfs.zfs.version.zpl 5 vfs.zfs.version.spa 5000 vfs.zfs.version.acl 1 vfs.zfs.version.ioctl 7 vfs.zfs.debug 0 vfs.zfs.super_owner 0 vfs.zfs.sync_pass_rewrite 2 vfs.zfs.sync_pass_dont_compress 5 vfs.zfs.sync_pass_deferred_free 2 vfs.zfs.zio.dva_throttle_enabled 1 vfs.zfs.zio.exclude_metadata 0 vfs.zfs.zio.use_uma 1 vfs.zfs.cache_flush_disable 0 vfs.zfs.zil_replay_disable 0 vfs.zfs.min_auto_ashift 12 vfs.zfs.max_auto_ashift 13 vfs.zfs.vdev.trim_max_pending 10000 vfs.zfs.vdev.bio_delete_disable 0 vfs.zfs.vdev.bio_flush_disable 0 vfs.zfs.vdev.queue_depth_pct 1000 vfs.zfs.vdev.write_gap_limit 4096 vfs.zfs.vdev.read_gap_limit 32768 vfs.zfs.vdev.aggregation_limit 131072 vfs.zfs.vdev.trim_max_active 64 vfs.zfs.vdev.trim_min_active 1 vfs.zfs.vdev.scrub_max_active 60 vfs.zfs.vdev.scrub_min_active 1 vfs.zfs.vdev.async_write_max_active 100 vfs.zfs.vdev.async_write_min_active 10 vfs.zfs.vdev.async_read_max_active 60 vfs.zfs.vdev.async_read_min_active 10 vfs.zfs.vdev.sync_write_max_active 200 vfs.zfs.vdev.sync_write_min_active 100 vfs.zfs.vdev.sync_read_max_active 100 vfs.zfs.vdev.sync_read_min_active 100 vfs.zfs.vdev.max_active 1000 vfs.zfs.vdev.async_write_active_max_dirty_percent60 vfs.zfs.vdev.async_write_active_min_dirty_percent30 vfs.zfs.vdev.mirror.non_rotating_seek_inc1 vfs.zfs.vdev.mirror.non_rotating_inc 0 vfs.zfs.vdev.mirror.rotating_seek_offset1048576 vfs.zfs.vdev.mirror.rotating_seek_inc 5 vfs.zfs.vdev.mirror.rotating_inc 0 vfs.zfs.vdev.trim_on_init 1 vfs.zfs.vdev.cache.bshift 16 vfs.zfs.vdev.cache.size 4194304 vfs.zfs.vdev.cache.max 65536 vfs.zfs.vdev.metaslabs_per_vdev 200 vfs.zfs.txg.timeout 4 vfs.zfs.space_map_blksz 4096 vfs.zfs.spa_min_slop 134217728 vfs.zfs.spa_slop_shift 5 vfs.zfs.spa_asize_inflation 24 vfs.zfs.deadman_enabled 0 vfs.zfs.deadman_checktime_ms 5000 vfs.zfs.deadman_synctime_ms 1000000 vfs.zfs.debug_flags 0 vfs.zfs.recover 0 vfs.zfs.spa_load_verify_data 1 vfs.zfs.spa_load_verify_metadata 1 vfs.zfs.spa_load_verify_maxinflight 10000 vfs.zfs.ccw_retry_interval 300 vfs.zfs.check_hostid 1 vfs.zfs.mg_fragmentation_threshold 85 vfs.zfs.mg_noalloc_threshold 0 vfs.zfs.condense_pct 200 vfs.zfs.metaslab.bias_enabled 1 vfs.zfs.metaslab.lba_weighting_enabled 1 vfs.zfs.metaslab.fragmentation_factor_enabled1 vfs.zfs.metaslab.preload_enabled 1 vfs.zfs.metaslab.preload_limit 3 vfs.zfs.metaslab.unload_delay 8 vfs.zfs.metaslab.load_pct 50 vfs.zfs.metaslab.min_alloc_size 33554432 vfs.zfs.metaslab.df_free_pct 4 vfs.zfs.metaslab.df_alloc_threshold 131072 vfs.zfs.metaslab.debug_unload 0 vfs.zfs.metaslab.debug_load 0 vfs.zfs.metaslab.fragmentation_threshold70 vfs.zfs.metaslab.gang_bang 16777217 vfs.zfs.free_bpobj_enabled 1 vfs.zfs.free_max_blocks -1 vfs.zfs.no_scrub_prefetch 0 vfs.zfs.no_scrub_io 0 vfs.zfs.resilver_min_time_ms 3000 vfs.zfs.free_min_time_ms 1000 vfs.zfs.scan_min_time_ms 1000 vfs.zfs.scan_idle 50 vfs.zfs.scrub_delay 4 vfs.zfs.resilver_delay 2 vfs.zfs.top_maxinflight 32 vfs.zfs.zfetch.array_rd_sz 1048576 vfs.zfs.zfetch.max_distance 8388608 vfs.zfs.zfetch.min_sec_reap 2 vfs.zfs.zfetch.max_streams 64 vfs.zfs.prefetch_disable 0 vfs.zfs.delay_scale 500000 vfs.zfs.delay_min_dirty_percent 60 vfs.zfs.dirty_data_sync 67108864 vfs.zfs.dirty_data_max_percent 10 vfs.zfs.dirty_data_max_max 4294967296 vfs.zfs.dirty_data_max 4294967296 vfs.zfs.max_recordsize 1048576 vfs.zfs.send_holes_without_birth_time 1 vfs.zfs.mdcomp_disable 0 vfs.zfs.nopwrite_enabled 1 vfs.zfs.dedup.prefetch 1 vfs.zfs.l2c_only_size 0 vfs.zfs.mfu_ghost_data_esize 62297529344 vfs.zfs.mfu_ghost_metadata_esize 0 vfs.zfs.mfu_ghost_size 62297529344 vfs.zfs.mfu_data_esize 66706433536 vfs.zfs.mfu_metadata_esize 2435194880 vfs.zfs.mfu_size 69802579968 vfs.zfs.mru_ghost_data_esize 60685495808 vfs.zfs.mru_ghost_metadata_esize 0 vfs.zfs.mru_ghost_size 60685495808 vfs.zfs.mru_data_esize 49709753856 vfs.zfs.mru_metadata_esize 1613580288 vfs.zfs.mru_size 51551468032 vfs.zfs.anon_data_esize 0 vfs.zfs.anon_metadata_esize 0 vfs.zfs.anon_size 6747136 vfs.zfs.l2arc_norw 0 vfs.zfs.l2arc_feed_again 1 vfs.zfs.l2arc_noprefetch 0 vfs.zfs.l2arc_feed_min_ms 200 vfs.zfs.l2arc_feed_secs 1 vfs.zfs.l2arc_headroom 32 vfs.zfs.l2arc_write_boost 268435456 vfs.zfs.l2arc_write_max 67108864 vfs.zfs.arc_meta_limit 77309411328 vfs.zfs.arc_free_target 254980 vfs.zfs.compressed_arc_enabled 1 vfs.zfs.arc_shrink_shift 7 vfs.zfs.arc_average_blocksize 8192 vfs.zfs.arc_min 18694015488 vfs.zfs.arc_max 149552123904 ------------------------------------------------------------------------ --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Feb 20 16:11:45 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C5651CE66D5 for ; Mon, 20 Feb 2017 16:11:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B4DCD1F79 for ; Mon, 20 Feb 2017 16:11:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1KGBiZb062839 for ; Mon, 20 Feb 2017 16:11:45 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Mon, 20 Feb 2017 16:11:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Feb 2017 16:11:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #13 from Andriy Gapon --- Created attachment 180167 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D180167&action= =3Dedit dtrace script (In reply to Lev A. Serebryakov from comment #11) Lev, assuming that you are still observing the condition, could you please = run the attached DTrace script for a few minutes and attach its output to this = bug? Also, is your kernel + zfs compiled with INVARIANTS? If not, it might be worthwhile enabling that option to increase chances of catching the bug. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Feb 21 08:45:43 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9BDFCE6DD8 for ; Tue, 21 Feb 2017 08:45:43 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 14E4D1E7 for ; Tue, 21 Feb 2017 08:45:42 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from webmail.norma.perm.ru (localhost [127.0.0.1]) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTP id v1L8jbec021578 for ; Tue, 21 Feb 2017 13:45:38 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1487666738; bh=khK6agRHIiTxmySavrq2kVtd8EAW+F1rwoq4FKEXeCw=; h=Date:From:To:Subject; b=jZ7RMEMNvgzxVtpzaxTqCS2WJ9h7+QV8rDFh5dPQk1nICbNTX44yFyZFg5PnVTmNx LvS+VA5rMnK3hFxkCK0K68ydw2PPdA3VTVEyMWI2UcqiznSQT0VQsMMqfGztjuVI51 FC8PCvNq6qVDH512lOjSP4p0Iyq4nhd+fY/Fi46g= MIME-Version: 1.0 Date: Tue, 21 Feb 2017 13:45:37 +0500 From: "Eugene M. Zheganin" To: freebsd-fs@freebsd.org Subject: zfs raidz overhead Message-ID: <1b54a2fe35407a95edca1f992fa08a71@norman-vivat.ru> X-Sender: emz@norma.perm.ru User-Agent: Roundcube Webmail/0.8.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Feb 2017 08:45:44 -0000 Hi. There's an interesting case described here: http://serverfault.com/questions/512018/strange-zfs-disk-space-usage-report-for-a-zvol [1] It's a user story who encountered that under some situations zfs on raidz could use up to 200% of the space for a zvol. I have also seen this. For instance: [root@san1:~]# zfs get volsize gamestop/reference1 NAME PROPERTY VALUE SOURCE gamestop/reference1 volsize 2,50T local [root@san1:~]# zfs get all gamestop/reference1 NAME PROPERTY VALUE SOURCE gamestop/reference1 type volume - gamestop/reference1 creation чт нояб. 24 9:09 2016 - gamestop/reference1 used 4,38T - gamestop/reference1 available 1,33T - gamestop/reference1 referenced 4,01T - gamestop/reference1 compressratio 1.00x - gamestop/reference1 reservation none default gamestop/reference1 volsize 2,50T local gamestop/reference1 volblocksize 8K - gamestop/reference1 checksum on default gamestop/reference1 compression off default gamestop/reference1 readonly off default gamestop/reference1 copies 1 default gamestop/reference1 refreservation none received gamestop/reference1 primarycache all default gamestop/reference1 secondarycache all default gamestop/reference1 usedbysnapshots 378G - gamestop/reference1 usedbydataset 4,01T - gamestop/reference1 usedbychildren 0 - gamestop/reference1 usedbyrefreservation 0 - gamestop/reference1 logbias latency default gamestop/reference1 dedup off default gamestop/reference1 mlslabel - gamestop/reference1 sync standard default gamestop/reference1 refcompressratio 1.00x - gamestop/reference1 written 4,89G - gamestop/reference1 logicalused 2,72T - gamestop/reference1 logicalreferenced 2,49T - gamestop/reference1 volmode default default gamestop/reference1 snapshot_limit none default gamestop/reference1 snapshot_count none default gamestop/reference1 redundant_metadata all default [root@san1:~]# zpool status gamestop pool: gamestop state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM gamestop ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 da8 ONLINE 0 0 0 da9 ONLINE 0 0 0 da11 ONLINE 0 0 0 errors: No known data errors or, another server (overhead in this case isn't that big, but still considerable): [root@san01:~]# zfs get all data/reference1 NAME PROPERTY VALUE SOURCE data/reference1 type volume - data/reference1 creation Fri Jan 6 11:23 2017 - data/reference1 used 3.82T - data/reference1 available 13.0T - data/reference1 referenced 3.22T - data/reference1 compressratio 1.00x - data/reference1 reservation none default data/reference1 volsize 2T local data/reference1 volblocksize 8K - data/reference1 checksum on default data/reference1 compression off default data/reference1 readonly off default data/reference1 copies 1 default data/reference1 refreservation none received data/reference1 primarycache all default data/reference1 secondarycache all default data/reference1 usedbysnapshots 612G - data/reference1 usedbydataset 3.22T - data/reference1 usedbychildren 0 - data/reference1 usedbyrefreservation 0 - data/reference1 logbias latency default data/reference1 dedup off default data/reference1 mlslabel - data/reference1 sync standard default data/reference1 refcompressratio 1.00x - data/reference1 written 498K - data/reference1 logicalused 2.37T - data/reference1 logicalreferenced 2.00T - data/reference1 volmode default default data/reference1 snapshot_limit none default data/reference1 snapshot_count none default data/reference1 redundant_metadata all default [root@san01:~]# zpool status gamestop pool: data state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da3 ONLINE 0 0 0 da4 ONLINE 0 0 0 da5 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 da8 ONLINE 0 0 0 da9 ONLINE 0 0 0 da10 ONLINE 0 0 0 da11 ONLINE 0 0 0 da12 ONLINE 0 0 0 raidz1-2 ONLINE 0 0 0 da13 ONLINE 0 0 0 da14 ONLINE 0 0 0 da15 ONLINE 0 0 0 da16 ONLINE 0 0 0 da17 ONLINE 0 0 0 errors: No known data errors So my question is - how to avoid it ? Right now I'm experimenting with the volblocksize, making it around 64k. I'm also suspecting that such overhead may be the subsequence of the various resizing operations, like extening the volsize of the volume or adding new disks into the pool, because I have a couple of servers with raidz where the initial disk/volsize configuration didn't change, and the referenced/volsize numbers are pretty much close to each other. Eugene. Links: ------ [1] http://serverfault.com/questions/512018/strange-zfs-disk-space-usage-report-for-a-zvol From owner-freebsd-fs@freebsd.org Tue Feb 21 09:25:06 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7937BCE60D5 for ; Tue, 21 Feb 2017 09:25:06 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (imap.slu.se [77.235.224.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA 3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 135FB1F0C for ; Tue, 21 Feb 2017 09:25:05 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by exch2-4.slu.se (77.235.224.124) with Microsoft SMTP Server (TLS) id 15.0.1236.3; Tue, 21 Feb 2017 10:09:52 +0100 Received: from exch2-4.slu.se ([fe80::e006:857e:1307:5005]) by exch2-4.slu.se ([fe80::e006:857e:1307:5005%22]) with mapi id 15.00.1236.000; Tue, 21 Feb 2017 10:09:52 +0100 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: "Eugene M. Zheganin" CC: "freebsd-fs@freebsd.org" Subject: Re: zfs raidz overhead Thread-Topic: zfs raidz overhead Thread-Index: AQHSjCJCwmvqtn7xL0CKHdy5p0XKYg== Date: Tue, 21 Feb 2017 09:09:52 +0000 Message-ID: <09aeee171ef34c1ba8fb7b4c684a7224@exch2-4.slu.se> Accept-Language: sv-SE, en-US Content-Language: sv-SE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Feb 2017 09:25:06 -0000 CkRlbiAyMSBmZWIuIDIwMTcgMDk6NDYgc2tyZXYgIkV1Z2VuZSBNLiBaaGVnYW5pbiIgPGVtekBu b3JtYS5wZXJtLnJ1PjoKPgo+Cj4KPiBIaS4gCj4KPiBUaGVyZSdzIGFuIGludGVyZXN0aW5nIGNh c2UgZGVzY3JpYmVkIGhlcmU6Cj4gaHR0cDovL3NlcnZlcmZhdWx0LmNvbS9xdWVzdGlvbnMvNTEy MDE4L3N0cmFuZ2UtemZzLWRpc2stc3BhY2UtdXNhZ2UtcmVwb3J0LWZvci1hLXp2b2wKPiBbMV0g Cj4KPiBJdCdzIGEgdXNlciBzdG9yeSB3aG8gZW5jb3VudGVyZWQgdGhhdCB1bmRlciBzb21lIHNp dHVhdGlvbnMgemZzIG9uCj4gcmFpZHogY291bGQgdXNlIHVwIHRvIDIwMCUgb2YgdGhlIHNwYWNl IGZvciBhIHp2b2wuIAo+Cj4gSSBoYXZlIGFsc28gc2VlbiB0aGlzLiBGb3IgaW5zdGFuY2U6IAo+ Cj4gW3Jvb3RAc2FuMTp+XSMgemZzIGdldCB2b2xzaXplIGdhbWVzdG9wL3JlZmVyZW5jZTEKPiDC oE5BTUUgUFJPUEVSVFkgVkFMVUUgU09VUkNFCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHZvbHNp emUgMiw1MFQgbG9jYWwKPiDCoFtyb290QHNhbjE6fl0jIHpmcyBnZXQgYWxsIGdhbWVzdG9wL3Jl ZmVyZW5jZTEgCj4gwqBOQU1FIFBST1BFUlRZIFZBTFVFIFNPVVJDRQo+IMKgZ2FtZXN0b3AvcmVm ZXJlbmNlMSB0eXBlIHZvbHVtZSAtCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIGNyZWF0aW9uINGH 0YIg0L3QvtGP0LEuIDI0IDk6MDkgMjAxNiAtCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHVzZWQg NCwzOFQgLQo+IMKgZ2FtZXN0b3AvcmVmZXJlbmNlMSBhdmFpbGFibGUgMSwzM1QgLQo+IMKgZ2Ft ZXN0b3AvcmVmZXJlbmNlMSByZWZlcmVuY2VkIDQsMDFUIC0KPiDCoGdhbWVzdG9wL3JlZmVyZW5j ZTEgY29tcHJlc3NyYXRpbyAxLjAweCAtCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHJlc2VydmF0 aW9uIG5vbmUgZGVmYXVsdAo+IMKgZ2FtZXN0b3AvcmVmZXJlbmNlMSB2b2xzaXplIDIsNTBUIGxv Y2FsCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHZvbGJsb2Nrc2l6ZSA4SyAtCj4gwqBnYW1lc3Rv cC9yZWZlcmVuY2UxIGNoZWNrc3VtIG9uIGRlZmF1bHQKPiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEg Y29tcHJlc3Npb24gb2ZmIGRlZmF1bHQKPiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEgcmVhZG9ubHkg b2ZmIGRlZmF1bHQKPiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEgY29waWVzIDEgZGVmYXVsdAo+IMKg Z2FtZXN0b3AvcmVmZXJlbmNlMSByZWZyZXNlcnZhdGlvbiBub25lIHJlY2VpdmVkCj4gwqBnYW1l c3RvcC9yZWZlcmVuY2UxIHByaW1hcnljYWNoZSBhbGwgZGVmYXVsdAo+IMKgZ2FtZXN0b3AvcmVm ZXJlbmNlMSBzZWNvbmRhcnljYWNoZSBhbGwgZGVmYXVsdAo+IMKgZ2FtZXN0b3AvcmVmZXJlbmNl MSB1c2VkYnlzbmFwc2hvdHMgMzc4RyAtCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHVzZWRieWRh dGFzZXQgNCwwMVQgLQo+IMKgZ2FtZXN0b3AvcmVmZXJlbmNlMSB1c2VkYnljaGlsZHJlbiAwIC0K PiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEgdXNlZGJ5cmVmcmVzZXJ2YXRpb24gMCAtCj4gwqBnYW1l c3RvcC9yZWZlcmVuY2UxIGxvZ2JpYXMgbGF0ZW5jeSBkZWZhdWx0Cj4gwqBnYW1lc3RvcC9yZWZl cmVuY2UxIGRlZHVwIG9mZiBkZWZhdWx0Cj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIG1sc2xhYmVs IC0KPiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEgc3luYyBzdGFuZGFyZCBkZWZhdWx0Cj4gwqBnYW1l c3RvcC9yZWZlcmVuY2UxIHJlZmNvbXByZXNzcmF0aW8gMS4wMHggLQo+IMKgZ2FtZXN0b3AvcmVm ZXJlbmNlMSB3cml0dGVuIDQsODlHIC0KPiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEgbG9naWNhbHVz ZWQgMiw3MlQgLQo+IMKgZ2FtZXN0b3AvcmVmZXJlbmNlMSBsb2dpY2FscmVmZXJlbmNlZCAyLDQ5 VCAtCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHZvbG1vZGUgZGVmYXVsdCBkZWZhdWx0Cj4gwqBn YW1lc3RvcC9yZWZlcmVuY2UxIHNuYXBzaG90X2xpbWl0IG5vbmUgZGVmYXVsdAo+IMKgZ2FtZXN0 b3AvcmVmZXJlbmNlMSBzbmFwc2hvdF9jb3VudCBub25lIGRlZmF1bHQKPiDCoGdhbWVzdG9wL3Jl ZmVyZW5jZTEgcmVkdW5kYW50X21ldGFkYXRhIGFsbCBkZWZhdWx0IAo+Cj4gW3Jvb3RAc2FuMTp+ XSMgenBvb2wgc3RhdHVzIGdhbWVzdG9wCj4gwqBwb29sOiBnYW1lc3RvcAo+IMKgc3RhdGU6IE9O TElORQo+IMKgc2Nhbjogbm9uZSByZXF1ZXN0ZWQKPiDCoGNvbmZpZzoKPgo+IMKgTkFNRSBTVEFU RSBSRUFEIFdSSVRFIENLU1VNCj4gwqBnYW1lc3RvcCBPTkxJTkUgMCAwIDAKPiDCoHJhaWR6MS0w IE9OTElORSAwIDAgMAo+IMKgZGE2IE9OTElORSAwIDAgMAo+IMKgZGE3IE9OTElORSAwIDAgMAo+ IMKgZGE4IE9OTElORSAwIDAgMAo+IMKgZGE5IE9OTElORSAwIDAgMAo+IMKgZGExMSBPTkxJTkUg MCAwIDAKPgo+IMKgZXJyb3JzOiBObyBrbm93biBkYXRhIGVycm9ycyAKPgo+IG9yLCBhbm90aGVy IHNlcnZlciAob3ZlcmhlYWQgaW4gdGhpcyBjYXNlIGlzbid0IHRoYXQgYmlnLCBidXQgc3RpbGwK PiBjb25zaWRlcmFibGUpOiAKPgo+IFtyb290QHNhbjAxOn5dIyB6ZnMgZ2V0IGFsbCBkYXRhL3Jl ZmVyZW5jZTEKPiDCoE5BTUUgUFJPUEVSVFkgVkFMVUUgU09VUkNFCj4gwqBkYXRhL3JlZmVyZW5j ZTEgdHlwZSB2b2x1bWUgLQo+IMKgZGF0YS9yZWZlcmVuY2UxIGNyZWF0aW9uIEZyaSBKYW4gNiAx MToyMyAyMDE3IC0KPiDCoGRhdGEvcmVmZXJlbmNlMSB1c2VkIDMuODJUIC0KPiDCoGRhdGEvcmVm ZXJlbmNlMSBhdmFpbGFibGUgMTMuMFQgLQo+IMKgZGF0YS9yZWZlcmVuY2UxIHJlZmVyZW5jZWQg My4yMlQgLQo+IMKgZGF0YS9yZWZlcmVuY2UxIGNvbXByZXNzcmF0aW8gMS4wMHggLQo+IMKgZGF0 YS9yZWZlcmVuY2UxIHJlc2VydmF0aW9uIG5vbmUgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2Ux IHZvbHNpemUgMlQgbG9jYWwKPiDCoGRhdGEvcmVmZXJlbmNlMSB2b2xibG9ja3NpemUgOEsgLQo+ IMKgZGF0YS9yZWZlcmVuY2UxIGNoZWNrc3VtIG9uIGRlZmF1bHQKPiDCoGRhdGEvcmVmZXJlbmNl MSBjb21wcmVzc2lvbiBvZmYgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2UxIHJlYWRvbmx5IG9m ZiBkZWZhdWx0Cj4gwqBkYXRhL3JlZmVyZW5jZTEgY29waWVzIDEgZGVmYXVsdAo+IMKgZGF0YS9y ZWZlcmVuY2UxIHJlZnJlc2VydmF0aW9uIG5vbmUgcmVjZWl2ZWQKPiDCoGRhdGEvcmVmZXJlbmNl MSBwcmltYXJ5Y2FjaGUgYWxsIGRlZmF1bHQKPiDCoGRhdGEvcmVmZXJlbmNlMSBzZWNvbmRhcnlj YWNoZSBhbGwgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2UxIHVzZWRieXNuYXBzaG90cyA2MTJH IC0KPiDCoGRhdGEvcmVmZXJlbmNlMSB1c2VkYnlkYXRhc2V0IDMuMjJUIC0KPiDCoGRhdGEvcmVm ZXJlbmNlMSB1c2VkYnljaGlsZHJlbiAwIC0KPiDCoGRhdGEvcmVmZXJlbmNlMSB1c2VkYnlyZWZy ZXNlcnZhdGlvbiAwIC0KPiDCoGRhdGEvcmVmZXJlbmNlMSBsb2diaWFzIGxhdGVuY3kgZGVmYXVs dAo+IMKgZGF0YS9yZWZlcmVuY2UxIGRlZHVwIG9mZiBkZWZhdWx0Cj4gwqBkYXRhL3JlZmVyZW5j ZTEgbWxzbGFiZWwgLQo+IMKgZGF0YS9yZWZlcmVuY2UxIHN5bmMgc3RhbmRhcmQgZGVmYXVsdAo+ IMKgZGF0YS9yZWZlcmVuY2UxIHJlZmNvbXByZXNzcmF0aW8gMS4wMHggLQo+IMKgZGF0YS9yZWZl cmVuY2UxIHdyaXR0ZW4gNDk4SyAtCj4gwqBkYXRhL3JlZmVyZW5jZTEgbG9naWNhbHVzZWQgMi4z N1QgLQo+IMKgZGF0YS9yZWZlcmVuY2UxIGxvZ2ljYWxyZWZlcmVuY2VkIDIuMDBUIC0KPiDCoGRh dGEvcmVmZXJlbmNlMSB2b2xtb2RlIGRlZmF1bHQgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2Ux IHNuYXBzaG90X2xpbWl0IG5vbmUgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2UxIHNuYXBzaG90 X2NvdW50IG5vbmUgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2UxIHJlZHVuZGFudF9tZXRhZGF0 YSBhbGwgZGVmYXVsdAo+IMKgW3Jvb3RAc2FuMDE6fl0jIHpwb29sIHN0YXR1cyBnYW1lc3RvcAo+ IMKgcG9vbDogZGF0YQo+IMKgc3RhdGU6IE9OTElORQo+IMKgc2Nhbjogbm9uZSByZXF1ZXN0ZWQK PiDCoGNvbmZpZzoKPgo+IMKgTkFNRSBTVEFURSBSRUFEIFdSSVRFIENLU1VNCj4gwqBkYXRhIE9O TElORSAwIDAgMAo+IMKgcmFpZHoxLTAgT05MSU5FIDAgMCAwCj4gwqBkYTMgT05MSU5FIDAgMCAw Cj4gwqBkYTQgT05MSU5FIDAgMCAwCj4gwqBkYTUgT05MSU5FIDAgMCAwCj4gwqBkYTYgT05MSU5F IDAgMCAwCj4gwqBkYTcgT05MSU5FIDAgMCAwCj4gwqByYWlkejEtMSBPTkxJTkUgMCAwIDAKPiDC oGRhOCBPTkxJTkUgMCAwIDAKPiDCoGRhOSBPTkxJTkUgMCAwIDAKPiDCoGRhMTAgT05MSU5FIDAg MCAwCj4gwqBkYTExIE9OTElORSAwIDAgMAo+IMKgZGExMiBPTkxJTkUgMCAwIDAKPiDCoHJhaWR6 MS0yIE9OTElORSAwIDAgMAo+IMKgZGExMyBPTkxJTkUgMCAwIDAKPiDCoGRhMTQgT05MSU5FIDAg MCAwCj4gwqBkYTE1IE9OTElORSAwIDAgMAo+IMKgZGExNiBPTkxJTkUgMCAwIDAKPiDCoGRhMTcg T05MSU5FIDAgMCAwCj4KPiDCoGVycm9yczogTm8ga25vd24gZGF0YSBlcnJvcnMgCj4KPiBTbyBt eSBxdWVzdGlvbiBpcyAtIGhvdyB0byBhdm9pZCBpdCA/IFJpZ2h0IG5vdyBJJ20gZXhwZXJpbWVu dGluZyB3aXRoCj4gdGhlIHZvbGJsb2Nrc2l6ZSwgbWFraW5nIGl0IGFyb3VuZCA2NGsuIEknbSBh bHNvIHN1c3BlY3RpbmcgdGhhdCBzdWNoCj4gb3ZlcmhlYWQgbWF5IGJlIHRoZSBzdWJzZXF1ZW5j ZSBvZiB0aGUgdmFyaW91cyByZXNpemluZyBvcGVyYXRpb25zLCBsaWtlCj4gZXh0ZW5pbmcgdGhl IHZvbHNpemUgb2YgdGhlIHZvbHVtZSBvciBhZGRpbmcgbmV3IGRpc2tzIGludG8gdGhlIHBvb2ws Cj4gYmVjYXVzZSBJIGhhdmUgYSBjb3VwbGUgb2Ygc2VydmVycyB3aXRoIHJhaWR6IHdoZXJlIHRo ZSBpbml0aWFsCj4gZGlzay92b2xzaXplIGNvbmZpZ3VyYXRpb24gZGlkbid0IGNoYW5nZSwgYW5k IHRoZSByZWZlcmVuY2VkL3ZvbHNpemUKPiBudW1iZXJzIGFyZSBwcmV0dHkgbXVjaCBjbG9zZSB0 byBlYWNoIG90aGVyLiAKPgo+IEV1Z2VuZS4gCj4KPiBMaW5rczoKPiAtLS0tLS0KPiBbMV0KPiBo dHRwOi8vc2VydmVyZmF1bHQuY29tL3F1ZXN0aW9ucy81MTIwMTgvc3RyYW5nZS16ZnMtZGlzay1z cGFjZS11c2FnZS1yZXBvcnQtZm9yLWEtenZvbAo+IF9fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fCj4gZnJlZWJzZC1mc0BmcmVlYnNkLm9yZyBtYWlsaW5nIGxp c3QKPiBodHRwczovL2xpc3RzLmZyZWVic2Qub3JnL21haWxtYW4vbGlzdGluZm8vZnJlZWJzZC1m cwo+IFRvIHVuc3Vic2NyaWJlLCBzZW5kIGFueSBtYWlsIHRvICJmcmVlYnNkLWZzLXVuc3Vic2Ny aWJlQGZyZWVic2Qub3JnIgoKSGVsbG8hCgpUaGlzIHByb2JsZW0gaXMgZmFyIGZyb20gbmV3LCBJ IGludmVzdGlnYXRlZCB0aGlzIGJhY2sgaW4gMjAxMyBhbmQgcG9zdGVkIG15IGZpbmRpbmdzIGhl cmU6Cmh0dHBzOi8vZm9ydW1zLmZyZWVic2Qub3JnL3RocmVhZHMvMzczNjUvCgovSw== From owner-freebsd-fs@freebsd.org Tue Feb 21 11:09:50 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1545CE8A45 for ; Tue, 21 Feb 2017 11:09:50 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 77A3D74F for ; Tue, 21 Feb 2017 11:09:50 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x234.google.com with SMTP id v186so106902944wmd.0 for ; Tue, 21 Feb 2017 03:09:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=K4r3PkIDy7sXfaM7FRIkieUwIXDKcdfQ4JvP6KJTtz0=; b=rbjBxumng7oiABIGJu8H/E/7K30etwZ0QltoowsaA4g9Ymw4HhPR6us+4f/a/omJMP Pr22q3k6EE/8pkgVa5JBHEB7VMUcClO/lerXlwdni+kwkqaY5QHyN4bGiReH5/xLHM3q yJt8owdCD8kZhOZ+Lk4618HRCPhxGQMdzDZF7LSf/5/uuJFs5IjDBWiDcM8D8k5Ok1I/ uiKKDZryZ2FjBzrawAA15N/FL+a16CEQa6OcdiCaMpCAKol2yGYm1dJ1aolguCCFAdhV j/3733sZXR9oa5v8EgpcY+r8pfXyuOJ4XB80H5AxlsUMYRIKkhaRb7j37cuMnBtBYlYg grCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=K4r3PkIDy7sXfaM7FRIkieUwIXDKcdfQ4JvP6KJTtz0=; b=uKaWT+gpMVCvmEvh9qNCe3C7dD97D37IRHxVuPzOmlQ1777voXNshn5YZgxueeiF4j fpf4O8RNx/vIyu5Y56EAotlzq4e+GD2G8TCeDhH2xx8F91HD7C+60UXyVR3pnREPuRry /+cGjyCml0EnBIfGTW9x1K/tlYhENCv0Ytswr/4agBpe5zPNlYIaFVkEHe+0ffMnI6h8 Z7dswmjsUWK23LfyUx936nDv5L5vapkDmD3uRnchFGYhL+816UOdywkht6yE0g4evRV5 m5j4nSZhDaqqA6Teu3/Fdj0jVUv9eYEgJfjt8LE/8iBPZrKPjXzC6nZUaRarF4kzYw56 Oq1A== X-Gm-Message-State: AMke39lwRN9nr8dlj5WGCfPeInGeaKm4lOldMFf+G3cTigjoHvWKM8J4vx3Ho0OjRWmdfabq X-Received: by 10.28.7.10 with SMTP id 10mr15081971wmh.55.1487675387750; Tue, 21 Feb 2017 03:09:47 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id h23sm19442131wrc.48.2017.02.21.03.09.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Feb 2017 03:09:47 -0800 (PST) Subject: Re: zfs raidz overhead To: freebsd-fs@freebsd.org References: <1b54a2fe35407a95edca1f992fa08a71@norman-vivat.ru> From: Steven Hartland Message-ID: <8cbb514b-92a9-c1c3-24e6-22cf9643ed97@multiplay.co.uk> Date: Tue, 21 Feb 2017 11:09:47 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: <1b54a2fe35407a95edca1f992fa08a71@norman-vivat.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Feb 2017 11:09:51 -0000 It doesn't directly address ZVOL's on RAIDZ but the following is a very good article from Matthew Ahrens on RAIDZ sizing: https://www.delphix.com/blog/delphix-engineering/zfs-raidz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz On 21/02/2017 08:45, Eugene M. Zheganin wrote: > > > Hi. > > There's an interesting case described here: > http://serverfault.com/questions/512018/strange-zfs-disk-space-usage-report-for-a-zvol > [1] > > It's a user story who encountered that under some situations zfs on > raidz could use up to 200% of the space for a zvol. > > I have also seen this. For instance: > > [root@san1:~]# zfs get volsize gamestop/reference1 > NAME PROPERTY VALUE SOURCE > gamestop/reference1 volsize 2,50T local > [root@san1:~]# zfs get all gamestop/reference1 > NAME PROPERTY VALUE SOURCE > gamestop/reference1 type volume - > gamestop/reference1 creation чт нояб. 24 9:09 2016 - > gamestop/reference1 used 4,38T - > gamestop/reference1 available 1,33T - > gamestop/reference1 referenced 4,01T - > gamestop/reference1 compressratio 1.00x - > gamestop/reference1 reservation none default > gamestop/reference1 volsize 2,50T local > gamestop/reference1 volblocksize 8K - > gamestop/reference1 checksum on default > gamestop/reference1 compression off default > gamestop/reference1 readonly off default > gamestop/reference1 copies 1 default > gamestop/reference1 refreservation none received > gamestop/reference1 primarycache all default > gamestop/reference1 secondarycache all default > gamestop/reference1 usedbysnapshots 378G - > gamestop/reference1 usedbydataset 4,01T - > gamestop/reference1 usedbychildren 0 - > gamestop/reference1 usedbyrefreservation 0 - > gamestop/reference1 logbias latency default > gamestop/reference1 dedup off default > gamestop/reference1 mlslabel - > gamestop/reference1 sync standard default > gamestop/reference1 refcompressratio 1.00x - > gamestop/reference1 written 4,89G - > gamestop/reference1 logicalused 2,72T - > gamestop/reference1 logicalreferenced 2,49T - > gamestop/reference1 volmode default default > gamestop/reference1 snapshot_limit none default > gamestop/reference1 snapshot_count none default > gamestop/reference1 redundant_metadata all default > > [root@san1:~]# zpool status gamestop > pool: gamestop > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > gamestop ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > da6 ONLINE 0 0 0 > da7 ONLINE 0 0 0 > da8 ONLINE 0 0 0 > da9 ONLINE 0 0 0 > da11 ONLINE 0 0 0 > > errors: No known data errors > > or, another server (overhead in this case isn't that big, but still > considerable): > > [root@san01:~]# zfs get all data/reference1 > NAME PROPERTY VALUE SOURCE > data/reference1 type volume - > data/reference1 creation Fri Jan 6 11:23 2017 - > data/reference1 used 3.82T - > data/reference1 available 13.0T - > data/reference1 referenced 3.22T - > data/reference1 compressratio 1.00x - > data/reference1 reservation none default > data/reference1 volsize 2T local > data/reference1 volblocksize 8K - > data/reference1 checksum on default > data/reference1 compression off default > data/reference1 readonly off default > data/reference1 copies 1 default > data/reference1 refreservation none received > data/reference1 primarycache all default > data/reference1 secondarycache all default > data/reference1 usedbysnapshots 612G - > data/reference1 usedbydataset 3.22T - > data/reference1 usedbychildren 0 - > data/reference1 usedbyrefreservation 0 - > data/reference1 logbias latency default > data/reference1 dedup off default > data/reference1 mlslabel - > data/reference1 sync standard default > data/reference1 refcompressratio 1.00x - > data/reference1 written 498K - > data/reference1 logicalused 2.37T - > data/reference1 logicalreferenced 2.00T - > data/reference1 volmode default default > data/reference1 snapshot_limit none default > data/reference1 snapshot_count none default > data/reference1 redundant_metadata all default > [root@san01:~]# zpool status gamestop > pool: data > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > data ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > da3 ONLINE 0 0 0 > da4 ONLINE 0 0 0 > da5 ONLINE 0 0 0 > da6 ONLINE 0 0 0 > da7 ONLINE 0 0 0 > raidz1-1 ONLINE 0 0 0 > da8 ONLINE 0 0 0 > da9 ONLINE 0 0 0 > da10 ONLINE 0 0 0 > da11 ONLINE 0 0 0 > da12 ONLINE 0 0 0 > raidz1-2 ONLINE 0 0 0 > da13 ONLINE 0 0 0 > da14 ONLINE 0 0 0 > da15 ONLINE 0 0 0 > da16 ONLINE 0 0 0 > da17 ONLINE 0 0 0 > > errors: No known data errors > > So my question is - how to avoid it ? Right now I'm experimenting with > the volblocksize, making it around 64k. I'm also suspecting that such > overhead may be the subsequence of the various resizing operations, like > extening the volsize of the volume or adding new disks into the pool, > because I have a couple of servers with raidz where the initial > disk/volsize configuration didn't change, and the referenced/volsize > numbers are pretty much close to each other. > > Eugene. > > Links: > ------ > [1] > http://serverfault.com/questions/512018/strange-zfs-disk-space-usage-report-for-a-zvol > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Tue Feb 21 14:13:17 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31953CE7720 for ; Tue, 21 Feb 2017 14:13:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 210B115DB for ; Tue, 21 Feb 2017 14:13:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1LEDACL015319 for ; Tue, 21 Feb 2017 14:13:17 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 209158] node / npm triggering zfs rename deadlock Date: Tue, 21 Feb 2017 14:13:10 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: xavi.garcia@gmail.com X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: avg@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Feb 2017 14:13:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D209158 Xavier Garcia changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |xavi.garcia@gmail.com --- Comment #49 from Xavier Garcia --- *** Bug 217267 has been marked as a duplicate of this bug. *** --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Tue Feb 21 18:12:17 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 125FBCE8B41 for ; Tue, 21 Feb 2017 18:12:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EDC83F5F for ; Tue, 21 Feb 2017 18:12:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1LICGH1031603 for ; Tue, 21 Feb 2017 18:12:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 217062] for file systems mounted with -o noexec, exec=off property does not work for mmap Date: Tue, 21 Feb 2017 18:12:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: brooks@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Feb 2017 18:12:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D217062 Brooks Davis changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |brooks@FreeBSD.org --- Comment #8 from Brooks Davis --- This seems like pointless consistency and seems likely to break existing (perhaps somewhat nonsensical) configurations. At first glance, it seems better for mprotect() to neglect to honor MNT_NOEXEC since, IMO, MNT_NOEXEC= is only really enforceable in the image activator. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Feb 21 23:31:34 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2085DCE800F for ; Tue, 21 Feb 2017 23:31:34 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DE34B14A8 for ; Tue, 21 Feb 2017 23:31:33 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: by mail-it0-x22a.google.com with SMTP id y135so70066801itc.1 for ; Tue, 21 Feb 2017 15:31:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=tpZgJL1bQvM1LLQ4nPUNNhKwHnUxgyz6IjlxUwPUisY=; b=ZU/waOu9Y7gdOXXAxQHaLeQ78JQM7d9CWNYnEvPvl8OtGrdpdKXJ+mA2TSsqIvoYdM /TpWbRms0LF6piwZ550XwEyfKs5Lcf0LIOya30BmzJXfUo0+YkzqLrOfzO16X2TT4aPc hw8jz/AC4r3ghp3V/Bm978juC8BFtHl5p5eT6Nchk/w0xXKKbP+rs1AFBfKaOyJUokso GBBR/vU/JSZ3pAMGWgZ0fIpecfk72ohMAcmKim1KqssbYZcm15b97dIjVFeJ5EBCRW1r skXANCzGfRb0Gi1k2WU/b7K3mD5z8XKKRY+PHo3R/UXtffHcyhnRmG4WCgcjSIf36glU SfkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=tpZgJL1bQvM1LLQ4nPUNNhKwHnUxgyz6IjlxUwPUisY=; b=d9T2lByM6o3hE0Tu3djViHi7M38GErJ5yCR3Ew4Y7sKpOpdJOGtp2n4JC46MPVAqQI m68xbF6kxjGaHdrfskFYnVn0jaNHyV2GKuv83eiSYKF/o3sGLnZK8eoqj2IfXc8M4e4M 5RKqpT62CpBUNCE8+sq6ZgngkTIBzUIuCW6C4YDiAWcBmA53Bnj6L+fi3py9ZDNGllVx ZuHjQ3WIbVq1L4uN8tlkIwZh1W5Gkd2fev9WPEsMnN2Z2Q9itYHhpZQIKh9AnBUVv8ZH Y0LBz8WD7e2xMEkqnKxXY77UjI07kT/zmtWHfe+nQit+fqlx0q3Dt/mtPxEIduQjtCSA MO5g== X-Gm-Message-State: AMke39mI/b2KEMLCbYY1yXeKJ2StvXiICAHRcem9LYDAhyNPUcmkMlSuxnpgt2QMUK9B7MNm3trd3/oxSj9D1w== X-Received: by 10.36.108.15 with SMTP id w15mr28914974itb.73.1487719892705; Tue, 21 Feb 2017 15:31:32 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.183.148 with HTTP; Tue, 21 Feb 2017 15:31:32 -0800 (PST) In-Reply-To: <1b54a2fe35407a95edca1f992fa08a71@norman-vivat.ru> References: <1b54a2fe35407a95edca1f992fa08a71@norman-vivat.ru> From: "Eric A. Borisch" Date: Tue, 21 Feb 2017 17:31:32 -0600 Message-ID: Subject: Re: zfs raidz overhead To: "Eugene M. Zheganin" Cc: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Feb 2017 23:31:34 -0000 On Tue, Feb 21, 2017 at 2:45 AM, Eugene M. Zheganin wrote: Hi. There's an interesting case described here: http://serverfault.com/questions/512018/strange-zfs-disk- space-usage-report-for-a-zvol [1] It's a user story who encountered that under some situations zfs on raidz could use up to 200% of the space for a zvol. I have also seen this. For instance: [root@san1:~]# zfs get volsize gamestop/reference1 NAME PROPERTY VALUE SOURCE gamestop/reference1 volsize 2,50T local [root@san1:~]# zfs get all gamestop/reference1 NAME PROPERTY VALUE SOURCE gamestop/reference1 type volume - gamestop/reference1 creation =D1=87=D1=82 =D0=BD=D0=BE=D1=8F=D0=B1. 24 9:0= 9 2016 - gamestop/reference1 used 4,38T - gamestop/reference1 available 1,33T - gamestop/reference1 referenced 4,01T - gamestop/reference1 compressratio 1.00x - gamestop/reference1 reservation none default gamestop/reference1 volsize 2,50T local gamestop/reference1 volblocksize 8K - gamestop/reference1 checksum on default gamestop/reference1 compression off default gamestop/reference1 readonly off default gamestop/reference1 copies 1 default gamestop/reference1 refreservation none received gamestop/reference1 primarycache all default gamestop/reference1 secondarycache all default gamestop/reference1 usedbysnapshots 378G - gamestop/reference1 usedbydataset 4,01T - gamestop/reference1 usedbychildren 0 - gamestop/reference1 usedbyrefreservation 0 - gamestop/reference1 logbias latency default gamestop/reference1 dedup off default gamestop/reference1 mlslabel - gamestop/reference1 sync standard default gamestop/reference1 refcompressratio 1.00x - gamestop/reference1 written 4,89G - gamestop/reference1 logicalused 2,72T - gamestop/reference1 logicalreferenced 2,49T - gamestop/reference1 volmode default default gamestop/reference1 snapshot_limit none default gamestop/reference1 snapshot_count none default gamestop/reference1 redundant_metadata all default [root@san1:~]# zpool status gamestop pool: gamestop state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM gamestop ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 da8 ONLINE 0 0 0 da9 ONLINE 0 0 0 da11 ONLINE 0 0 0 errors: No known data errors or, another server (overhead in this case isn't that big, but still considerable): [root@san01:~]# zfs get all data/reference1 NAME PROPERTY VALUE SOURCE data/reference1 type volume - data/reference1 creation Fri Jan 6 11:23 2017 - data/reference1 used 3.82T - data/reference1 available 13.0T - data/reference1 referenced 3.22T - data/reference1 compressratio 1.00x - data/reference1 reservation none default data/reference1 volsize 2T local data/reference1 volblocksize 8K - data/reference1 checksum on default data/reference1 compression off default data/reference1 readonly off default data/reference1 copies 1 default data/reference1 refreservation none received data/reference1 primarycache all default data/reference1 secondarycache all default data/reference1 usedbysnapshots 612G - data/reference1 usedbydataset 3.22T - data/reference1 usedbychildren 0 - data/reference1 usedbyrefreservation 0 - data/reference1 logbias latency default data/reference1 dedup off default data/reference1 mlslabel - data/reference1 sync standard default data/reference1 refcompressratio 1.00x - data/reference1 written 498K - data/reference1 logicalused 2.37T - data/reference1 logicalreferenced 2.00T - data/reference1 volmode default default data/reference1 snapshot_limit none default data/reference1 snapshot_count none default data/reference1 redundant_metadata all default [root@san01:~]# zpool status gamestop pool: data state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da3 ONLINE 0 0 0 da4 ONLINE 0 0 0 da5 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 da8 ONLINE 0 0 0 da9 ONLINE 0 0 0 da10 ONLINE 0 0 0 da11 ONLINE 0 0 0 da12 ONLINE 0 0 0 raidz1-2 ONLINE 0 0 0 da13 ONLINE 0 0 0 da14 ONLINE 0 0 0 da15 ONLINE 0 0 0 da16 ONLINE 0 0 0 da17 ONLINE 0 0 0 errors: No known data errors So my question is - how to avoid it ? Right now I'm experimenting with the volblocksize, making it around 64k. I'm also suspecting that such overhead may be the subsequence of the various resizing operations, like extening the volsize of the volume or adding new disks into the pool, because I have a couple of servers with raidz where the initial disk/volsize configuration didn't change, and the referenced/volsize numbers are pretty much close to each other. Eugene. Links: ------ [1] http://serverfault.com/questions/512018/strange-zfs-disk- space-usage-report-for-a-zvol It comes down to the zpool's sector size (2^ashift) and the volblocksize -- I'm guessing your old servers are at ashift=3D9 (512), and the new one is a= t 12 (4096), likely with 4k drives. This is the smallest/atomic size of reads & writes to a drive from ZFS. As described in [1]: * Allocations need to be a multiple of (p+1) sectors, where p is your parity level; for raidz1, p=3D=3D1, and allocations need to be in multiples= of (1+1)=3D2 sectors, or 8k (for ashift=3D12; this is the physical size / alignment on drive). * It also needs to have enough parity for failures, so it also depends [2] on number of drives in pool at larger block/record sizes. So considering those requirements, and your zvol with volblocksize=3D8k and compression=3Doff, allocations for one logical 8k block are always composed physically of two (4k) data sectors, one (p=3D1) parity sector (4k), and on= e padding sector (4k) to satisfy being a multiple of (p+1=3D) 2, or 16k (allocated on disk space), hence your observed 2x data size being actually allocated. Each of these blocks will be on a different drive. This is different from the sector-level parity in RAID5 As Matthew Ahrens [1] points out: "Note that setting a small recordsize with 4KB sector devices results in universally poor space efficiency -- RAIDZ-p is no better than p-way mirrors for recordsize=3D4K or 8K." Things you can do: * Use ashift=3D9 (and perhaps 512-byte sector drives). The same layout rul= es still apply, but now your 'atomic' size is 512b. You will want to test performance. * Use a larger volblocksize, especially if the filesystem on the zvol uses a larger block size. If you aren't performance sensitive, use a larger volblocksize even if the hosted filesystem doesn't. (But test this out to see how performance sensitive you really are! ;) You'll need to use something like dd to move data between different block size zvols. * Enable compression if the contents are compressible (some likely will be.) * Use a pool created from mirrors instead of raidz if you need high-performance small blocks while retaining redundancy. You don't get efficient (better than mirrors) redundancy, performant small (as in small multiple of zpool's sector size) block sizes, and zfs's flexibility all at once. - Eric [1] https://www.delphix.com/blog/delphix-engineering/zfs-rai dz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz [2] My spin on Ahren's spreadsheet: https://docs.google.com/spread sheets/d/13sJPc6ZW6_441vWAUiSvKMReJW4z34Ix5JSs44YXRyM/edit?usp=3Dsharing From owner-freebsd-fs@freebsd.org Wed Feb 22 04:59:43 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C604FCE9FDA for ; Wed, 22 Feb 2017 04:59:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B53BC12C7 for ; Wed, 22 Feb 2017 04:59:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1M4xgnr055724 for ; Wed, 22 Feb 2017 04:59:43 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 217062] for file systems mounted with -o noexec, exec=off property does not work for mmap Date: Wed, 22 Feb 2017 04:59:43 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 04:59:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D217062 --- Comment #9 from Konstantin Belousov --- (In reply to Brooks Davis from comment #8) I already stated that noexec is only meaningful as a protection against ima= ge activator, but another argument for consistency is other OSes behaviour. In particular, I verified that Linux behaves in a way requested by the bug reporter, before I committed the r313967. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 09:55:04 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8AE01CE8E71 for ; Wed, 22 Feb 2017 09:55:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7AAB2C5E for ; Wed, 22 Feb 2017 09:55:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1M9t1UZ037658 for ; Wed, 22 Feb 2017 09:55:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 09:55:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 09:55:04 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #14 from Lev A. Serebryakov --- (In reply to Andriy Gapon from comment #13) No INVARIANTS, but I'll add them today. And I'll run script and return with results. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 10:00:09 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A287CE8051 for ; Wed, 22 Feb 2017 10:00:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8A051ECF for ; Wed, 22 Feb 2017 10:00:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MA09XI047987 for ; Wed, 22 Feb 2017 10:00:09 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 10:00:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 10:00:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #15 from Lev A. Serebryakov --- Looks like dtrace -s doesn't like this->tail =3D (arc_buf_hdr_t *)((char*)this->tail_ - this->offset); and this->head =3D (arc_buf_hdr_t *)((char*)this->head_ - this->offset); Should I add some include files? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 10:28:15 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB91FCE8DFB for ; Wed, 22 Feb 2017 10:28:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CB51B3B0 for ; Wed, 22 Feb 2017 10:28:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MASF1Q022235 for ; Wed, 22 Feb 2017 10:28:15 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 10:28:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 10:28:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #16 from Andriy Gapon --- You mean it doesn't like the types? Try writing them as `arc_buf_hdr_t (note the leading backtick). --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 10:42:19 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7900CE90B4 for ; Wed, 22 Feb 2017 10:42:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B75AFD9A for ; Wed, 22 Feb 2017 10:42:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MAgJrZ061465 for ; Wed, 22 Feb 2017 10:42:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 10:42:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 10:42:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #17 from Lev A. Serebryakov --- Error is: dtrace: failed to compile script l2arc.d: line 13: syntax error near ")" If I comment-out line 13 same is on line 15. Parenthesis look balanced for = me! Backtick doesn't help. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 11:57:25 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7267CE8733 for ; Wed, 22 Feb 2017 11:57:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A64B813B5 for ; Wed, 22 Feb 2017 11:57:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MBvPgQ045319 for ; Wed, 22 Feb 2017 11:57:25 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 11:57:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 11:57:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #18 from Andriy Gapon --- Please try to write them as zfs.ko`arc_buf_hdr_t or kernel`arc_buf_hdr_t. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 12:06:14 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 917ACCE93D7 for ; Wed, 22 Feb 2017 12:06:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 814BC1E13 for ; Wed, 22 Feb 2017 12:06:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MC6EDn088068 for ; Wed, 22 Feb 2017 12:06:14 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 12:06:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 12:06:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #19 from Lev A. Serebryakov --- Same error, both with zfs.ko` and kernel` --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 12:14:11 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35EB8CE9673 for ; Wed, 22 Feb 2017 12:14:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1B3E231D for ; Wed, 22 Feb 2017 12:14:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MCE9cq000532 for ; Wed, 22 Feb 2017 12:14:10 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 12:14:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 12:14:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #20 from Andriy Gapon --- Can you see arc_buf_hdr_t in output of 'ctfdump -t /boot/kernel/zfs.ko' ? How about arc_buf_hdr? Could you try to use (struct zfs.ko`arc_buf_hdr *) instead of (arc_buf_hdr_t *)? I assume you use ZFS as a module? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 12:31:48 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 852D0CE9ED9 for ; Wed, 22 Feb 2017 12:31:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 74D46FDA for ; Wed, 22 Feb 2017 12:31:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MCVmCR079794 for ; Wed, 22 Feb 2017 12:31:48 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 12:31:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 12:31:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #21 from Lev A. Serebryakov --- Yes, ZFS is used as module. % kldstat Id Refs Address Size Name 1 50 0xffffffff80200000 d7afc0 kernel 2 1 0xffffffff80f7c000 300088 zfs.ko 3 11 0xffffffff8127d000 ab00 opensolaris.ko 4 1 0xffffffff81411000 3318b linux.ko 5 3 0xffffffff81445000 2b9e linux_common.ko 6 1 0xffffffff81448000 2e845 linux64.ko 7 1 0xffffffff81477000 4192 linprocfs.ko 8 1 0xffffffff8147c000 357 dtraceall.ko 9 9 0xffffffff8147d000 389c6 dtrace.ko 10 1 0xffffffff814b6000 623 dtmalloc.ko 11 1 0xffffffff814b7000 18ba dtnfscl.ko 12 1 0xffffffff814b9000 1dcb fbt.ko 13 1 0xffffffff814bb000 531c1 fasttrap.ko 14 1 0xffffffff8150f000 b9f sdt.ko 15 1 0xffffffff81510000 6dc4 systrace.ko 16 1 0xffffffff81517000 6d24 systrace_freebsd32.ko 17 1 0xffffffff8151e000 fb7 profile.ko % ctfdump -t /boot/kernel/zfs.ko /boot/kernel/zfs.ko does not contain .SUNW_ctf data % cat /etc/src.conf BATCH_DELETE_OLD_FILES=3Dyes WITHOUT_TESTS=3Dyes % "struct zfs.ko`arc_buf_hdr" gives antoher error: dtrace: failed to compile script l2arc.d: line 1: probe description fbt::l2arc_write_buffers:entry does not match any probes % > sudo dtrace -l | grep l2arc 27809 fbt zfs l2arc_do_free_on_write entry 27810 fbt zfs l2arc_evict entry 27811 fbt zfs l2arc_evict return 27812 fbt zfs l2arc_feed_thread entry 27813 fbt zfs l2arc_read_done entry 27814 fbt zfs l2arc_write_done entry 29273 fbt zfs l2arc_init entry 29731 fbt zfs l2arc_stop entry 30261 fbt zfs l2arc_remove_vdev entry 31154 fbt zfs l2arc_vdev_present entry 31155 fbt zfs l2arc_vdev_present return 31189 fbt zfs l2arc_fini entry 32113 fbt zfs l2arc_start entry 32114 fbt zfs l2arc_start return 32334 fbt zfs l2arc_add_vdev entry 34552 sdt zfs none l2arc-= hit 34553 sdt zfs none l2arc-= read 34554 sdt zfs none l2arc-= miss 34565 sdt zfs none l2arc-evict 34566 sdt zfs none l2arc-write 34567 sdt zfs none l2arc-iodone % Should it be "l2arc_write_done"? But there is no "l2arc__write_done:return"! --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 13:09:25 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0BAC9CE965F for ; Wed, 22 Feb 2017 13:09:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E1A701E0D for ; Wed, 22 Feb 2017 13:09:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MD9Mf4040339 for ; Wed, 22 Feb 2017 13:09:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 13:09:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 13:09:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #22 from Andriy Gapon --- (In reply to Lev A. Serebryakov from comment #21) > Should it be "l2arc_write_done"? No... the function probably got inlined. Sorry then, I can't help you with this DTrace script. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 13:11:49 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47DD8CE9817 for ; Wed, 22 Feb 2017 13:11:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 37805D2 for ; Wed, 22 Feb 2017 13:11:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MDBnox050557 for ; Wed, 22 Feb 2017 13:11:49 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 13:11:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 13:11:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #23 from Lev A. Serebryakov --- I could rebuild module with debug options and without optimization=E2=80=A6 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 16:29:15 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D78ECE9DA4 for ; Wed, 22 Feb 2017 16:29:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6CC971D0F for ; Wed, 22 Feb 2017 16:29:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MGTFTL051561 for ; Wed, 22 Feb 2017 16:29:15 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 217062] for file systems mounted with -o noexec, exec=off property does not work for mmap Date: Wed, 22 Feb 2017 16:29:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: brooks@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 16:29:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D217062 --- Comment #10 from Brooks Davis --- (In reply to Konstantin Belousov from comment #9) The fact that linux does this is indeed a good argument for doing the same.= =20 Thanks for pointing that out. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 16:35:16 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71292CE901A for ; Wed, 22 Feb 2017 16:35:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 60BF2208 for ; Wed, 22 Feb 2017 16:35:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MGZCdM069929 for ; Wed, 22 Feb 2017 16:35:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 209158] node / npm triggering zfs rename deadlock Date: Wed, 22 Feb 2017 16:35:12 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: grembo@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: avg@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 16:35:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D209158 Michael Gmelin changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |grembo@FreeBSD.org --- Comment #50 from Michael Gmelin --- Is there a chance to get this into 10.3-RELEASE-pXX? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 18:53:52 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72B14CE9F64 for ; Wed, 22 Feb 2017 18:53:52 +0000 (UTC) (envelope-from rhuang.work@gmail.com) Received: from mail-pg0-x234.google.com (mail-pg0-x234.google.com [IPv6:2607:f8b0:400e:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 43BC11DE8 for ; Wed, 22 Feb 2017 18:53:52 +0000 (UTC) (envelope-from rhuang.work@gmail.com) Received: by mail-pg0-x234.google.com with SMTP id s67so4306254pgb.3 for ; Wed, 22 Feb 2017 10:53:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:mime-version:subject:message-id:date:to; bh=VLZFzqxqPjGuYDAf+6k0ZxBtg7PBk3IG1r52fJAMKAE=; b=L4TvfnduU6zhFwNJwx8GCl34Tr5B8cu86histhY5633lcajQwetBDWtd0s7iDl2YBh ggX6oS7CKvloQ0g4soBjYfqFeTP955QHHlqmD5FVJ7OhMAT6LQWnpKjdCLDxqM1EVHD1 rQxAoxKcnVRZtXV4rf3rWqoJB42nWIj+KbXEgoCG1G1/aust3C6lv1AD7XjvWlgtHJi3 ljcovciv6+G9GkEHSiszFDds0uhpQLG1XyRvNf2/FgnSrl3Yr07PofnyVCqb/gSNWHUZ eHNG7FNYFgFI7KhNZfGmxY6K6Nh2PMjUbtP7dzY0kUmnKRgtNgv2EU8o2hakOybU3koP igUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:date:to; bh=VLZFzqxqPjGuYDAf+6k0ZxBtg7PBk3IG1r52fJAMKAE=; b=el/YIosMXX4I1ij+jFI1s33BwQu4cJH/ZBgf+r06ejZAvy5CtOlot+StEC0abTBh5A 1jOndFXWeTZ5r4mEa4Pz8ipfh/HSb4SuSD3LwzSePgYlM1nHrUP2x/xgB7iA5Zmroc9Y TDhn8g24cpN3J/URobLiSRe6ZJoyMH4ZFV6MBBjs0gwPbRVrJp6FT4FR3PTfbBuQVxLj vmLDwTUgnsagVTcwO11BOKgfqwLZJCbS1+9PEuR0XFc5zNgT8Pgsv1hOlDxOOh7kLpH4 8ZhXwuUuxNYaMp3nU6aKu7h4FNBnYxFqxpfkoxh4e7jHqU3rv+0W7X1YFWe3hc/Xsd6Z eoug== X-Gm-Message-State: AMke39my6blqdO7+8VE9oVM4yTBtlsKE4M8QJoigv4rxuKFOA7+tbUKy0rZ+MsUiHYzFEQ== X-Received: by 10.84.133.97 with SMTP id 88mr44866984plf.87.1487789631543; Wed, 22 Feb 2017 10:53:51 -0800 (PST) Received: from ?IPv6:2605:e000:1c0d:196:e8ac:ac92:7f7d:65c4? ([2605:e000:1c0d:196:e8ac:ac92:7f7d:65c4]) by smtp.gmail.com with ESMTPSA id 2sm5049132pfv.100.2017.02.22.10.53.50 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 Feb 2017 10:53:50 -0800 (PST) From: Ricky Huang Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: 10.2 zfs-on-root will not boot after an unclean reboot Message-Id: Date: Wed, 22 Feb 2017 10:53:49 -0800 To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3259) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 18:53:52 -0000 Hello all, Server got power-cycled by the datacenter (accident) and now it won't = boot. Previous =E2=80=9Cclear" reboots have always been fine. It's a ZFS-on-root setup installed via FreeBSD installer and it's now = displaying: > FreeBSD/x86 boot > ZFS: I/O error - blocks larger than 128K are not supported > ZFS: can't find dataset u >=20 Any suggestions on where I can begin to diagnose this issue? Note: I have infrequent access to the server, so please "batch" the = suggestions so I can try them when I make my way into the d.c. Thanks in advance! From owner-freebsd-fs@freebsd.org Wed Feb 22 20:30:37 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD750CE913D for ; Wed, 22 Feb 2017 20:30:37 +0000 (UTC) (envelope-from eric@lumaforge.com) Received: from mail-oi0-x22c.google.com (mail-oi0-x22c.google.com [IPv6:2607:f8b0:4003:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7BB6CE4C for ; Wed, 22 Feb 2017 20:30:37 +0000 (UTC) (envelope-from eric@lumaforge.com) Received: by mail-oi0-x22c.google.com with SMTP id 2so7708743oif.0 for ; Wed, 22 Feb 2017 12:30:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lumaforge-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:cc; bh=7NyeRuvy5X41ZuWHXUQ/aZJJMFyCbmB9ElhjlB+ltRo=; b=bcAw+X1DYWIrpMKjWUt88Ks7nb5PZyT2f3sUIm8rTdAIPIAbS/t0dYT9Q2wQhdNv/e /rPM/bAPhiOyyrtFoiHWEHBScfXKjJnjpGHkJwMF1Cq9kYgsZEpjShgGR/D+I86zj6em Bjkva/GBTd7/WOiaLLFdt+OMXfPsmOuicHGczrjnG2h/Z8JIL4iRb9hCoPMRgCcNfnSD rqZazltJWDKx4GdTHDw9C3qJ3pn11RaN1C53vHnrNh+TSWZ+NqNkGNN1miwAsLac4nbn thMBTXNXH0JL9n+naTjAtYbwcdz3TBl0Z9S6GihxDILHrxRPGS1W085kDybHwwFR/Cvy DuYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:cc; bh=7NyeRuvy5X41ZuWHXUQ/aZJJMFyCbmB9ElhjlB+ltRo=; b=kpYPvIiIAh7AnvsORkBX3N3m8uIlcOKaF81Mw60x+7ClJJ2q4pH7C0pfV+SKHgfwfn te4ZIYQucYd04YZe9iTfns9aQgBzu+uUg4Pt+eI4W4Q0nqHUdte/V+2AaViCB37/xp4A lwG7bYtf/mkrh24Wlgp0s90BVsZpr8D3WQo+M4jIh3aP8/9C3Ze+L8qJ8BUiu/g58mfZ J8Maei0AM1tDHnMdjm0RFXyaIJmmfX9e0otsiPDeTFNdW95imEJAa41HmtwmypNxR8IB UHHmaDPY/yMqOLoEx4yzUrz4mSPw4HsNMufInKphKKQx/42SPzfHVbf98jD13zRy27/n s/bg== X-Gm-Message-State: AMke39kbj2jfQ8ucoYAM29WcZSvigL0f3lcnH9wmy6tjeSXVQAEgoIYmbpbJSWp2uoVFtpUWvIs0KxjtIjYbzg== X-Received: by 10.202.178.11 with SMTP id b11mr19312991oif.101.1487795436349; Wed, 22 Feb 2017 12:30:36 -0800 (PST) MIME-Version: 1.0 Received: by 10.182.230.195 with HTTP; Wed, 22 Feb 2017 12:30:35 -0800 (PST) Received: by 10.182.230.195 with HTTP; Wed, 22 Feb 2017 12:30:35 -0800 (PST) In-Reply-To: References: From: Eric Altman Date: Wed, 22 Feb 2017 12:30:35 -0800 Message-ID: Subject: Re: 10.2 zfs-on-root will not boot after an unclean reboot Cc: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 20:30:37 -0000 Generally when I see those kinds of boot errors, I check hardware first. Especially after hard shutdowns. Check that BIOS settings (particularly on your HBA) were not affected. Make sure all the drives are showing up. Might need to boot from a Live USB and check the pool's status and/or repair the ZFS cache. Run a scrub if you can import but not boot from it. On Feb 22, 2017 10:53 AM, "Ricky Huang" wrote: > Hello all, > > Server got power-cycled by the datacenter (accident) and now it won't > boot. Previous =E2=80=9Cclear" reboots have always been fine. > > It's a ZFS-on-root setup installed via FreeBSD installer and it's now > displaying: > > > FreeBSD/x86 boot > > ZFS: I/O error - blocks larger than 128K are not supported > > ZFS: can't find dataset u > > > > Any suggestions on where I can begin to diagnose this issue? > > Note: I have infrequent access to the server, so please "batch" the > suggestions so I can try them when I make my way into the d.c. > > > Thanks in advance! > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Wed Feb 22 20:36:20 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF104CE95C0 for ; Wed, 22 Feb 2017 20:36:20 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 67ADF169C for ; Wed, 22 Feb 2017 20:36:20 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x229.google.com with SMTP id v186so151775383wmd.0 for ; Wed, 22 Feb 2017 12:36:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=+HtC8kbEqCoPYI/UYwsbTL7O9JPpuhWex+jospeWPMQ=; b=Hv2bTR5RUVdSzataz/Rfs6sARuU0v+XbRTIalFjYPzbrH0Zs0D5i6MHAuQ7b1l/mAS ZXVZlLxoq3JmCXo9VHpV9OkJ+/JFlOg8yJo390eVFGpi7p/AHK70WTPcrz/Y0xjzNhvM 8IAhNp70aZ/Jx06x+vuyslvPb4W9UA9cVNdu12rPwZDq07gZUQ01KxXXO1sUJgi0ns+W 6M8k1AjFsD+LherNxuvI2CCUjK6CVIknEkYuETRBJS2GHUN4zJ1rEmbb20F5+7Pd2RYn /Ukp67gLkNc2lvnXqmIdqCq0Ag3YykBlyUqzgrX+f84YCRa3YDyX6hXB58dttDjz4JXQ c0qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=+HtC8kbEqCoPYI/UYwsbTL7O9JPpuhWex+jospeWPMQ=; b=tBEs1pvGQH5NOmBofAzYMwRL8KHSos1AfvHtpq+wZi1cdu73q4fKg6woYL8PoZSQEs x6GoM78xYtst/T7hnwWkJ7jdfC5rKHmAB/lNrVSZv1dg4wsjtbgiOd3tqqLxLlpUw34h wlldoPHSkbJaSVcJdiNKLgKn1V0f/otHCIj/BZ2hAnjSKt0Gqog7lZBsUm85hl96elqT B2D5M2mf6+/zmbW+R7jY2s25Vi4I9Mrh7eS2IttLc/IVtb/28nsXbM9l79oyCGIygx/q qI38Sg156WbsM23APvx3xd2i5Ydz6F3yju0pAEV1+nvb2IJakS8KVBJm7wLQpsWTUna0 Hfow== X-Gm-Message-State: AMke39mZCJoWuv0f9qOJ9xR9/gmXoeqUNx9UeXVi7OSL9bUOGeGcVvMuUtDXBafxMb1+4NjV X-Received: by 10.28.62.144 with SMTP id l138mr131466wma.50.1487795778343; Wed, 22 Feb 2017 12:36:18 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.4]) by smtp.gmail.com with ESMTPSA id 202sm3426075wmp.20.2017.02.22.12.36.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 Feb 2017 12:36:17 -0800 (PST) Subject: Re: 10.2 zfs-on-root will not boot after an unclean reboot To: freebsd-fs@freebsd.org References: From: Steven Hartland Message-ID: Date: Wed, 22 Feb 2017 20:36:17 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 20:36:20 -0000 On 22/02/2017 18:53, Ricky Huang wrote: > Hello all, > > Server got power-cycled by the datacenter (accident) and now it won't boot. Previous “clear" reboots have always been fine. > > It's a ZFS-on-root setup installed via FreeBSD installer and it's now displaying: > >> FreeBSD/x86 boot >> ZFS: I/O error - blocks larger than 128K are not supported >> ZFS: can't find dataset u >> > Any suggestions on where I can begin to diagnose this issue? > > Note: I have infrequent access to the server, so please "batch" the suggestions so I can try them when I make my way into the d.c. > > Given the "blocks larger than 128k are not supported" message it sounds like you enabled large block support on your root pool, which is not supported. If this is the case then your only option is to boot from a rescue disk and transfer the data off. Note if this the case, its not the unclean reboot that caused the issue its the large block size, which would have happened even with a clean reboot. Regards Steve From owner-freebsd-fs@freebsd.org Wed Feb 22 20:47:57 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EC76CE9A4A; Wed, 22 Feb 2017 20:47:57 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) by mx1.freebsd.org (Postfix) with ESMTP id 2F041D4; Wed, 22 Feb 2017 20:47:57 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:e0f4:994:662:862]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 5217D339; Wed, 22 Feb 2017 23:47:55 +0300 (MSK) Date: Wed, 22 Feb 2017 23:47:42 +0300 From: Lev Serebryakov Reply-To: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <492944119.20170222234742@serebryakov.spb.ru> To: freebsd-stable@freebsd.org, freebsd-fs@freebsd.org Subject: Is it known problem, that zfs.ko could not be built with system compiler (clang 3.9.1) without optimization? MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 20:47:57 -0000 Hello Freebsd-stable, Now if you build zfs.ko with -O0 it panics on boot. If you use default optimization level, a lot of fbt DTreace probes are missing. -- Best regards, Lev mailto:lev@FreeBSD.org From owner-freebsd-fs@freebsd.org Wed Feb 22 20:52:15 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C33D1CE9EAB; Wed, 22 Feb 2017 20:52:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-yw0-x22c.google.com (mail-yw0-x22c.google.com [IPv6:2607:f8b0:4002:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8298B995; Wed, 22 Feb 2017 20:52:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-yw0-x22c.google.com with SMTP id p77so7573825ywg.1; Wed, 22 Feb 2017 12:52:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=cPktHW3sIWytEwQbcd5QsAeR8dJQlSWi99ee3bFg/lk=; b=lqn1Azs3Gj3LCU5DTi4vb7AzTgV/azU+p02lFUe2oSZyH4MFZrhFcmyx61p6pN7DOH 4pYCcWNZjP7M73BeZCESaq9bUqRREscwxJMK6nYcNPD2J4bSFg3k0quOKiPLqMX8JKdc 8hrDl0xITmAnxukzXV0Tk673euRKxiOh2EP16pRvm05EvI8nbb/9hOoMzJiTZy8VJkVR 2hAQaAFHhAM/AnDAXC7VOf1GuP7xrmn62H3PrS9gOS8ylQvKAOhNZSRjpnDhwHDBs1Nr XIk7lv/2L7cg6xQZ8Y2ovr0CmyHGwJOHXtkZ1XLFOA9D7b003nnGEVhucF7zUzoxRj9t YSng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=cPktHW3sIWytEwQbcd5QsAeR8dJQlSWi99ee3bFg/lk=; b=KibRTXzryc+Pn71ZfQmn3tNstLuPZwtBvhAtpmfEHUNrCs8D4wbWcr7PtuBmyhWApF AX2aMVJy2vs4T39ouqtW6COBIrgDmIX5tdpUgNLMFlykmX8CNFj7Y1y7OtgWOMAJzVYo aZKYj/ClfldaZrKHLBfIExacq6lXEkIELn0doMRlzMSyudWD1g6rUbFxm/ycme3qZdSK QKRYFG07It/qThtTjPLXUlzc0ykdmTDGymWKBB+HfwUmD+HlduMYo7jQHntNw0vip9UD NCbahOVJS4w7IxckMWnwb6jVwbVw6PevPFmhFS5+tLkxFQelpcEp+xNh11JaV+We6uQe vdKg== X-Gm-Message-State: AMke39mAwaKmUKj91+8c468RBrC5pEMC1TkygBab3+rRrChfcXxmorTvD+Ajn9EVUs/n4UWYbkVDh/u5O8KSWQ== X-Received: by 10.129.68.31 with SMTP id r31mr56684ywa.307.1487796734453; Wed, 22 Feb 2017 12:52:14 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.129.38.133 with HTTP; Wed, 22 Feb 2017 12:52:14 -0800 (PST) In-Reply-To: <492944119.20170222234742@serebryakov.spb.ru> References: <492944119.20170222234742@serebryakov.spb.ru> From: Alan Somers Date: Wed, 22 Feb 2017 13:52:14 -0700 X-Google-Sender-Auth: r8JtiaJY8EEMpz9eiBG9Ei-2eNU Message-ID: Subject: Re: Is it known problem, that zfs.ko could not be built with system compiler (clang 3.9.1) without optimization? To: Lev Serebryakov Cc: FreeBSD , freebsd-fs Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 20:52:15 -0000 The panic is definitely a bug. You should create a bug on bugzilla for that one, if it isn't there already. It's to be expected that a lot of FBT probes won't be present in the default build. But there are two ways to ameliorate that: 1) Add a "__noinline" attribute to any function you want to trace 2) Add "#pragma clang optimize off" and "#pragma clang optimize on" around the code you want to debug. Note that even with optimization disabled, clang may still choose to inline static functions. But it does make stack traces easier to debug. -Alan On Wed, Feb 22, 2017 at 1:47 PM, Lev Serebryakov wrote: > Hello Freebsd-stable, > > Now if you build zfs.ko with -O0 it panics on boot. > > If you use default optimization level, a lot of fbt DTreace probes are > missing. > > -- > Best regards, > Lev mailto:lev@FreeBSD.org > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Wed Feb 22 21:06:35 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F9C6CE9527; Wed, 22 Feb 2017 21:06:35 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 025D01214; Wed, 22 Feb 2017 21:06:35 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cge7L-000O9x-Cb; Thu, 23 Feb 2017 00:06:31 +0300 Date: Thu, 23 Feb 2017 00:06:31 +0300 From: Slawa Olhovchenkov To: Lev Serebryakov Cc: freebsd-stable@freebsd.org, freebsd-fs@freebsd.org Subject: Re: Is it known problem, that zfs.ko could not be built with system compiler (clang 3.9.1) without optimization? Message-ID: <20170222210631.GH15630@zxy.spb.ru> References: <492944119.20170222234742@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <492944119.20170222234742@serebryakov.spb.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 21:06:35 -0000 On Wed, Feb 22, 2017 at 11:47:42PM +0300, Lev Serebryakov wrote: > Hello Freebsd-stable, > > Now if you build zfs.ko with -O0 it panics on boot. > > If you use default optimization level, a lot of fbt DTreace probes are > missing. Is this related to http://llvm.org/bugs/show_bug.cgi?id=18420 ? From owner-freebsd-fs@freebsd.org Wed Feb 22 21:13:38 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04A2CCE99A9; Wed, 22 Feb 2017 21:13:38 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 96ABC1AB4; Wed, 22 Feb 2017 21:13:36 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id v1ML5jP3012590 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 22 Feb 2017 22:05:46 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: lev@FreeBSD.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id v1ML5g01037465 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 23 Feb 2017 04:05:42 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Is it known problem, that zfs.ko could not be built with system compiler (clang 3.9.1) without optimization? To: Lev Serebryakov , freebsd-stable@freebsd.org, freebsd-fs@freebsd.org References: <492944119.20170222234742@serebryakov.spb.ru> From: Eugene Grosbein Message-ID: <58ADFD21.1010208@grosbein.net> Date: Thu, 23 Feb 2017 04:05:37 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <492944119.20170222234742@serebryakov.spb.ru> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 21:13:38 -0000 23.02.2017 3:47, Lev Serebryakov пишет: > Hello Freebsd-stable, > > Now if you build zfs.ko with -O0 it panics on boot. > > If you use default optimization level, a lot of fbt DTreace probes are > missing. If you use it with i386 (32 bits), you must use loader.conf tunnable: kern.kstack_pages=4 or corresponding kernel options KSTACK_PAGES=4, do you? From owner-freebsd-fs@freebsd.org Wed Feb 22 21:50:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EA63CEA537 for ; Wed, 22 Feb 2017 21:50:05 +0000 (UTC) (envelope-from bsd@vink.pl) Received: from mail-qt0-x22a.google.com (mail-qt0-x22a.google.com [IPv6:2607:f8b0:400d:c0d::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C0A95FA8 for ; Wed, 22 Feb 2017 21:50:04 +0000 (UTC) (envelope-from bsd@vink.pl) Received: by mail-qt0-x22a.google.com with SMTP id k15so14539135qtg.3 for ; Wed, 22 Feb 2017 13:50:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vink-pl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=TtIleobm99Y2GJcmokq5ON09IgvEfOUND7fifvSNL+g=; b=hB5wMhPYO+D2I1+koi5/Hz9JolqF8pigePum19IhZeoGBbxdBNNWY5VXTtLimfVNe6 yap4ZApZMt5pNjEyPsVpawNK7q+M1DhGmDTzLGARHyrmZ/XafDmhGKHOA+fbgYwTReG3 Gki6s6OlPcg0jFIU/gRwVBGC6dpWb2PntxJAPjnOa810N3Ybf+j+x87DKqg2j/WZVP6Y d2USruXW4bVKN6g7ZfnrFIJmIwD6yNGmeSPZMa2Qq5e14GrUv93IDxzMD2FrvFO/imB1 zdI39N8F9jnDBO9+EeKTmpOpYvCIM8OBGb9HkFfKziU/eRkHUt2qpWEmpNFn8dvOyuHt pjIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=TtIleobm99Y2GJcmokq5ON09IgvEfOUND7fifvSNL+g=; b=dMFJc0XjGllowx4JbtP04kyg3BD9u9KU3jD8TjRuTz/mdZf6C7ruOtYCBrkIznqnX7 nIN8Oz9V8GhhT92Rrp//ULdW8Vs/gLb8OGCMn27WFCeq9rGFLwPqbQuJiYn6rnI391LJ uc2tiTjFEQ07ZpEeBJwo4+9EawjeWpg2yX/uSnY97tJtqI2h65l5THV0EIVK2VGF9Xhk ROr0IxmLl2c0UBpP9JO+UP2ZGjnfALqBubvF3dpJQaMZQQpX3XPDahciFVtXApM3Fht/ YjXj50lZQHKQWo/L9F+pHK+uMXQINK60yQ+0SUARfR2hICcESeXkferaT7E27AHh5TSw /NBw== X-Gm-Message-State: AMke39kvlEBkTlLgxQOFuKX6DuoLV+tDUbwrUVuCLE4ISLUYY6eSLVBbZkfLeYDcUJ/v0g== X-Received: by 10.200.38.196 with SMTP id 4mr31003087qtp.96.1487800203173; Wed, 22 Feb 2017 13:50:03 -0800 (PST) Received: from mail-qk0-f172.google.com (mail-qk0-f172.google.com. [209.85.220.172]) by smtp.gmail.com with ESMTPSA id 102sm1488374qkx.49.2017.02.22.13.50.02 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 Feb 2017 13:50:02 -0800 (PST) Received: by mail-qk0-f172.google.com with SMTP id x71so15957480qkb.3 for ; Wed, 22 Feb 2017 13:50:02 -0800 (PST) X-Received: by 10.55.201.27 with SMTP id q27mr35796548qki.296.1487800202064; Wed, 22 Feb 2017 13:50:02 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.148.170 with HTTP; Wed, 22 Feb 2017 13:50:01 -0800 (PST) In-Reply-To: References: <1b54a2fe35407a95edca1f992fa08a71@norman-vivat.ru> From: Wiktor Niesiobedzki Date: Wed, 22 Feb 2017 22:50:01 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: zfs raidz overhead To: "Eric A. Borisch" Cc: "Eugene M. Zheganin" , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 21:50:05 -0000 I can add to this, that this is not only seen on raidz, but also on mirror pools, such as this: # zpool status tank pool: tank state: ONLINE scan: scrub repaired 0 in 3h22m with 0 errors on Thu Feb 9 06:47:07 2017 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/tank1.eli ONLINE 0 0 0 gpt/tank2.eli ONLINE 0 0 0 errors: No known data errors When I createted test zvols: # zfs create -V10gb -o volblocksize=3D8k tank/tst-8k # zfs create -V10gb -o volblocksize=3D16k tank/tst-16k # zfs create -V10gb -o volblocksize=3D32k tank/tst-32k # zfs create -V10gb -o volblocksize=3D64k tank/tst-64k # zfs create -V10gb -o volblocksize=3D128k tank/tst-128k # zfs get used tank/tst-8k NAME PROPERTY VALUE SOURCE tank/tst-8k used 10.3G - root@kadlubek:~ # zfs get used tank/tst-16k NAME PROPERTY VALUE SOURCE tank/tst-16k used 10.2G - root@kadlubek:~ # zfs get used tank/tst-32k NAME PROPERTY VALUE SOURCE tank/tst-32k used 10.1G - root@kadlubek:~ # zfs get used tank/tst-64k NAME PROPERTY VALUE SOURCE tank/tst-64k used 10.0G - root@kadlubek:~ # zfs get used tank/tst-128k NAME PROPERTY VALUE SOURCE tank/tst-128k used 10.0G - root@kadlubek:~ # So it might be related not only to raidz pools. I also noted, that snapshots impact used stats far much, than usedbysnapshot value: zfs get volsize,used,referenced,compressratio,volblocksize,usedbysnapshots,= usedbydataset,usedbychildren tank/dkr-thinpool NAME PROPERTY VALUE SOURCE tank/dkr-thinpool volsize 10G local tank/dkr-thinpool used 12.0G - tank/dkr-thinpool referenced 1.87G - tank/dkr-thinpool compressratio 1.91x - tank/dkr-thinpool volblocksize 64K - tank/dkr-thinpool usedbysnapshots 90.4M - tank/dkr-thinpool usedbydataset 1.87G - tank/dkr-thinpool usedbychildren 0 - On a 10G volume, filled with 2G of data, and 90M used by snapshosts, used is 2G. When I destroy the snapshots, used will drop to 10.0G. Cheers, Wiktor 2017-02-22 0:31 GMT+01:00 Eric A. Borisch : > On Tue, Feb 21, 2017 at 2:45 AM, Eugene M. Zheganin > wrote: > > > > Hi. > > There's an interesting case described here: > http://serverfault.com/questions/512018/strange-zfs-disk- > space-usage-report-for-a-zvol > [1] > > It's a user story who encountered that under some situations zfs on > raidz could use up to 200% of the space for a zvol. > > I have also seen this. For instance: > > [root@san1:~]# zfs get volsize gamestop/reference1 > NAME PROPERTY VALUE SOURCE > gamestop/reference1 volsize 2,50T local > [root@san1:~]# zfs get all gamestop/reference1 > NAME PROPERTY VALUE SOURCE > gamestop/reference1 type volume - > gamestop/reference1 creation =D1=87=D1=82 =D0=BD=D0=BE=D1=8F=D0=B1. 24 9= :09 2016 - > gamestop/reference1 used 4,38T - > gamestop/reference1 available 1,33T - > gamestop/reference1 referenced 4,01T - > gamestop/reference1 compressratio 1.00x - > gamestop/reference1 reservation none default > gamestop/reference1 volsize 2,50T local > gamestop/reference1 volblocksize 8K - > gamestop/reference1 checksum on default > gamestop/reference1 compression off default > gamestop/reference1 readonly off default > gamestop/reference1 copies 1 default > gamestop/reference1 refreservation none received > gamestop/reference1 primarycache all default > gamestop/reference1 secondarycache all default > gamestop/reference1 usedbysnapshots 378G - > gamestop/reference1 usedbydataset 4,01T - > gamestop/reference1 usedbychildren 0 - > gamestop/reference1 usedbyrefreservation 0 - > gamestop/reference1 logbias latency default > gamestop/reference1 dedup off default > gamestop/reference1 mlslabel - > gamestop/reference1 sync standard default > gamestop/reference1 refcompressratio 1.00x - > gamestop/reference1 written 4,89G - > gamestop/reference1 logicalused 2,72T - > gamestop/reference1 logicalreferenced 2,49T - > gamestop/reference1 volmode default default > gamestop/reference1 snapshot_limit none default > gamestop/reference1 snapshot_count none default > gamestop/reference1 redundant_metadata all default > > [root@san1:~]# zpool status gamestop > pool: gamestop > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > gamestop ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > da6 ONLINE 0 0 0 > da7 ONLINE 0 0 0 > da8 ONLINE 0 0 0 > da9 ONLINE 0 0 0 > da11 ONLINE 0 0 0 > > errors: No known data errors > > or, another server (overhead in this case isn't that big, but still > considerable): > > [root@san01:~]# zfs get all data/reference1 > NAME PROPERTY VALUE SOURCE > data/reference1 type volume - > data/reference1 creation Fri Jan 6 11:23 2017 - > data/reference1 used 3.82T - > data/reference1 available 13.0T - > data/reference1 referenced 3.22T - > data/reference1 compressratio 1.00x - > data/reference1 reservation none default > data/reference1 volsize 2T local > data/reference1 volblocksize 8K - > data/reference1 checksum on default > data/reference1 compression off default > data/reference1 readonly off default > data/reference1 copies 1 default > data/reference1 refreservation none received > data/reference1 primarycache all default > data/reference1 secondarycache all default > data/reference1 usedbysnapshots 612G - > data/reference1 usedbydataset 3.22T - > data/reference1 usedbychildren 0 - > data/reference1 usedbyrefreservation 0 - > data/reference1 logbias latency default > data/reference1 dedup off default > data/reference1 mlslabel - > data/reference1 sync standard default > data/reference1 refcompressratio 1.00x - > data/reference1 written 498K - > data/reference1 logicalused 2.37T - > data/reference1 logicalreferenced 2.00T - > data/reference1 volmode default default > data/reference1 snapshot_limit none default > data/reference1 snapshot_count none default > data/reference1 redundant_metadata all default > [root@san01:~]# zpool status gamestop > pool: data > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > data ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > da3 ONLINE 0 0 0 > da4 ONLINE 0 0 0 > da5 ONLINE 0 0 0 > da6 ONLINE 0 0 0 > da7 ONLINE 0 0 0 > raidz1-1 ONLINE 0 0 0 > da8 ONLINE 0 0 0 > da9 ONLINE 0 0 0 > da10 ONLINE 0 0 0 > da11 ONLINE 0 0 0 > da12 ONLINE 0 0 0 > raidz1-2 ONLINE 0 0 0 > da13 ONLINE 0 0 0 > da14 ONLINE 0 0 0 > da15 ONLINE 0 0 0 > da16 ONLINE 0 0 0 > da17 ONLINE 0 0 0 > > errors: No known data errors > > So my question is - how to avoid it ? Right now I'm experimenting with > the volblocksize, making it around 64k. I'm also suspecting that such > overhead may be the subsequence of the various resizing operations, like > extening the volsize of the volume or adding new disks into the pool, > because I have a couple of servers with raidz where the initial > disk/volsize configuration didn't change, and the referenced/volsize > numbers are pretty much close to each other. > > Eugene. > > Links: > ------ > [1] > http://serverfault.com/questions/512018/strange-zfs-disk- > space-usage-report-for-a-zvol > > > It comes down to the zpool's sector size (2^ashift) and the volblocksize = -- > I'm guessing your old servers are at ashift=3D9 (512), and the new one is= at > 12 (4096), likely with 4k drives. This is the smallest/atomic size of rea= ds > & writes to a drive from ZFS. > > As described in [1]: > * Allocations need to be a multiple of (p+1) sectors, where p is your > parity level; for raidz1, p=3D=3D1, and allocations need to be in multipl= es of > (1+1)=3D2 sectors, or 8k (for ashift=3D12; this is the physical size / > alignment on drive). > * It also needs to have enough parity for failures, so it also depends [= 2] > on number of drives in pool at larger block/record sizes. > > So considering those requirements, and your zvol with volblocksize=3D8k a= nd > compression=3Doff, allocations for one logical 8k block are always compos= ed > physically of two (4k) data sectors, one (p=3D1) parity sector (4k), and = one > padding sector (4k) to satisfy being a multiple of (p+1=3D) 2, or 16k > (allocated on disk space), hence your observed 2x data size being actuall= y > allocated. Each of these blocks will be on a different drive. This is > different from the sector-level parity in RAID5 > > As Matthew Ahrens [1] points out: "Note that setting a small recordsize > with 4KB sector devices results in universally poor space efficiency -- > RAIDZ-p is no better than p-way mirrors for recordsize=3D4K or 8K." > > Things you can do: > > * Use ashift=3D9 (and perhaps 512-byte sector drives). The same layout r= ules > still apply, but now your 'atomic' size is 512b. You will want to test > performance. > * Use a larger volblocksize, especially if the filesystem on the zvol us= es > a larger block size. If you aren't performance sensitive, use a larger > volblocksize even if the hosted filesystem doesn't. (But test this out to > see how performance sensitive you really are! ;) You'll need to use > something like dd to move data between different block size zvols. > * Enable compression if the contents are compressible (some likely will > be.) > * Use a pool created from mirrors instead of raidz if you need > high-performance small blocks while retaining redundancy. > > You don't get efficient (better than mirrors) redundancy, performant smal= l > (as in small multiple of zpool's sector size) block sizes, and zfs's > flexibility all at once. > > - Eric > > [1] https://www.delphix.com/blog/delphix-engineering/zfs-rai > dz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz > [2] My spin on Ahren's spreadsheet: https://docs.google.com/spread > sheets/d/13sJPc6ZW6_441vWAUiSvKMReJW4z34Ix5JSs44YXRyM/edit?usp=3Dsharing > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Wed Feb 22 21:58:35 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69A2FCEA73F; Wed, 22 Feb 2017 21:58:35 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 52F2214A8; Wed, 22 Feb 2017 21:58:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA28153; Wed, 22 Feb 2017 23:58:31 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1cgevf-000OBM-Na; Wed, 22 Feb 2017 23:58:31 +0200 Subject: Re: Is it known problem, that zfs.ko could not be built with system compiler (clang 3.9.1) without optimization? To: Lev Serebryakov , freebsd-stable@FreeBSD.org, freebsd-fs@FreeBSD.org References: <492944119.20170222234742@serebryakov.spb.ru> From: Andriy Gapon Message-ID: <85ad9000-b585-35e7-ccf1-6601ac4c062c@FreeBSD.org> Date: Wed, 22 Feb 2017 23:57:35 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: <492944119.20170222234742@serebryakov.spb.ru> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 21:58:35 -0000 On 22/02/2017 22:47, Lev Serebryakov wrote: > Hello Freebsd-stable, > > Now if you build zfs.ko with -O0 it panics on boot. I have seen a problem that matches your description, but not necessarily the same one: https://lists.freebsd.org/pipermail/freebsd-hackers/2016-July/049768.html > If you use default optimization level, a lot of fbt DTreace probes are > missing. You can try several different (orthogonal, even) approaches: - compile ZFS into the kernel - use higher optimization level, but add -fno-optimize-sibling-calls to CLFAGS - pkg install amd64-xtoolchain-gcc and then use CROSS_TOOLCHAIN=amd64-gcc command line argument for buildkernel -- Andriy Gapon From owner-freebsd-fs@freebsd.org Wed Feb 22 22:02:22 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 942E3CEA9C4 for ; Wed, 22 Feb 2017 22:02:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 83EAD19BC for ; Wed, 22 Feb 2017 22:02:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MM2Ldl011296 for ; Wed, 22 Feb 2017 22:02:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 22:02:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 22:02:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #24 from Lev A. Serebryakov --- Ok, dtrace is no option for now, but INVARIANTS give me panic very quickly: Panic String: solaris assert: write_psize <=3D target_sz (0x803000 <=3D 0x800000), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ar= c.c, line: 7140 I have core & all this stuff, of course, and could provide additional info.= But for now I'm turning off L2ARC as I need stable system/ --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 22:42:07 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D9F3CE900B for ; Wed, 22 Feb 2017 22:42:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 67D84F21 for ; Wed, 22 Feb 2017 22:42:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MMg5lk007586 for ; Wed, 22 Feb 2017 22:42:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 22:42:06 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 22:42:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #25 from Andriy Gapon --- (In reply to Lev A. Serebryakov from comment #24) Could you please send me a copy of your arc.c file exactly as it is now in = your source tree? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 22:48:18 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8E06CE9261 for ; Wed, 22 Feb 2017 22:48:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D8DFA175A for ; Wed, 22 Feb 2017 22:48:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MMmIMf017335 for ; Wed, 22 Feb 2017 22:48:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 22:48:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 22:48:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #26 from Lev A. Serebryakov --- Sent by e-mail. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Feb 22 23:26:41 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFB83CE9E57 for ; Wed, 22 Feb 2017 23:26:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DF1CCFD0 for ; Wed, 22 Feb 2017 23:26:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1MNQeT5033145 for ; Wed, 22 Feb 2017 23:26:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Wed, 22 Feb 2017 23:26:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 23:26:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #179917|0 |1 is obsolete| | --- Comment #27 from Andriy Gapon --- Created attachment 180235 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D180235&action= =3Dedit the patch Re-upload the patch to try to fix dos new lines. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Feb 23 00:49:45 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 524C5CE738B for ; Thu, 23 Feb 2017 00:49:45 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23ACF1CFB for ; Thu, 23 Feb 2017 00:49:45 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: by mail-it0-x235.google.com with SMTP id 203so235740ith.0 for ; Wed, 22 Feb 2017 16:49:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=qQVQfCQ8Y6p23O+/N+wIbrVB58bfzOKWQqNBpQ34XXk=; b=ivmyJhjgxh9HJ2Lqp6RRLnDSm3mNklzUop318JzQ8Gjy8+oyPPaa9zCi90jbjcsR+Z n55K95OpCBVIcXkaRZH6cUnF5RotVbvNjIWIjSdOgrwtfQjXjGDIw/9fFPF68Qa2EVRX tPhUIoc87DXU2OLnyOWK7g1VA1tEaEgIayjWMka8xQvpn9wXtk2b5naNpoQCdirGl6cb 1AUrcBCoMYMgjxcsQYu1w+km5wzZ8VaOoxGw491p4XE6/+2zCUcBoipuLG0uqIBVIgYD lfNe+we/7vRCU9cjYH/Aj76aE6sT29f+VIE8eOHPmAxoWgOMrCxh4U6n1D0QZvgYyI45 CNkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=qQVQfCQ8Y6p23O+/N+wIbrVB58bfzOKWQqNBpQ34XXk=; b=kWdD6BNRDk4EZRgH6g+vejU0VvVgukrLvyxAbvreX579GLk+xpoxkBtvi+M7z1KHIp uSTqUwhKlJwnlgr7XU6fCQJEn48Qzipyx4Yl90vSbGLIRBO/lA0MKfvM9R3WNHlP8XKh Vozg+zdsJyVlx1/K6LMt/Rx4a4G7Og0r8tC+gvP9Q/M+6WpOSkD725JWSYxcI9JveD+g kyf9KbvjZNvK9xX56w88tCwdiMesjmZrf91vuI68LE5TS3RnjWw0nJ/spkHIHKXB4fjp Rg9soeSgFEiQLVe1ewpAFyjlYA+t8s5pwya1V8OUg1+ohYx5xb1ExKMMMjFEwF4xzpNy Rg1A== X-Gm-Message-State: AMke39ms03SuP7tsTlw5pzfxunoTB1VXVdF7/ws8OQ70kd4QAJIm3qxjpSR9+iTAinTxA7N+zsigYHkmVjtb4A== X-Received: by 10.107.3.195 with SMTP id e64mr9275037ioi.18.1487810984437; Wed, 22 Feb 2017 16:49:44 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.183.148 with HTTP; Wed, 22 Feb 2017 16:49:43 -0800 (PST) In-Reply-To: References: <1b54a2fe35407a95edca1f992fa08a71@norman-vivat.ru> From: "Eric A. Borisch" Date: Wed, 22 Feb 2017 18:49:43 -0600 Message-ID: Subject: Re: zfs raidz overhead To: Wiktor Niesiobedzki Cc: "Eugene M. Zheganin" , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2017 00:49:45 -0000 On Wed, Feb 22, 2017 at 3:50 PM, Wiktor Niesiobedzki wrote: > I can add to this, that this is not only seen on raidz, but also on > mirror pools, such as this: > # zpool status tank > pool: tank > state: ONLINE > scan: scrub repaired 0 in 3h22m with 0 errors on Thu Feb 9 06:47:07 2017 > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > gpt/tank1.eli ONLINE 0 0 0 > gpt/tank2.eli ONLINE 0 0 0 > > errors: No known data errors > > > When I createted test zvols: > # zfs create -V10gb -o volblocksize=8k tank/tst-8k > # zfs create -V10gb -o volblocksize=16k tank/tst-16k > # zfs create -V10gb -o volblocksize=32k tank/tst-32k > # zfs create -V10gb -o volblocksize=64k tank/tst-64k > # zfs create -V10gb -o volblocksize=128k tank/tst-128k > > # zfs get used tank/tst-8k > NAME PROPERTY VALUE SOURCE > tank/tst-8k used 10.3G - > root@kadlubek:~ # zfs get used tank/tst-16k > NAME PROPERTY VALUE SOURCE > tank/tst-16k used 10.2G - > root@kadlubek:~ # zfs get used tank/tst-32k > NAME PROPERTY VALUE SOURCE > tank/tst-32k used 10.1G - > root@kadlubek:~ # zfs get used tank/tst-64k > NAME PROPERTY VALUE SOURCE > tank/tst-64k used 10.0G - > root@kadlubek:~ # zfs get used tank/tst-128k > NAME PROPERTY VALUE SOURCE > tank/tst-128k used 10.0G - Nope, that all looks correct. There is space reserved for metadata (checksum for example) when you create a zvol. Since checksums are computed by block, it makes sense that more metadata is required for the 8k volblocksize. > I also noted, that snapshots impact used stats far much, than usedbysnapshot value: > zfs get volsize,used,referenced,compressratio,volblocksize, > usedbysnapshots,usedbydataset,usedbychildren > tank/dkr-thinpool > NAME PROPERTY VALUE SOURCE > tank/dkr-thinpool volsize 10G local > tank/dkr-thinpool used 12.0G - > tank/dkr-thinpool referenced 1.87G - > tank/dkr-thinpool compressratio 1.91x - > tank/dkr-thinpool volblocksize 64K - > tank/dkr-thinpool usedbysnapshots 90.4M - > tank/dkr-thinpool usedbydataset 1.87G - > tank/dkr-thinpool usedbychildren 0 - > > > On a 10G volume, filled with 2G of data, and 90M used by snapshosts, > used is 2G. When I destroy the snapshots, used will drop to 10.0G. > It makes sense when you just remember that 'used' is enforcing a contract to the user that 10G of new data can be written to the zvol (refreservation, I'm sure it's set to 10G here), and still have all of the current snapshots will still be available. Breaking that down, usedbysnapshots refers to the data that is no longer referenced by the dataset (overwritten, in the case of a zvol.) Used is the space the the dataset is consuming (i.e. not available for other datasets). This means that 10G (refreservation)+ the referenced property of your earliest snapshot + the written value of all but your latest snapshot + the aforementioned metadata overhead needs to be reserved, or 'used' by the dataset. (I think I got that calculation right.) So even though usedbysnapshots is 90M (90M of blocks have been overwritten), the referenced and written properties of the snapshot accumulate to the space you're seeing here. Once all the snapshots are gone, used drops to (almost) the volsize. Hope that helps. If you want thin provisioning, you can unset the refreservation, or just create the volume with the '-s' option. - Eric From owner-freebsd-fs@freebsd.org Thu Feb 23 07:27:36 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB454CEAB4B for ; Thu, 23 Feb 2017 07:27:36 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1749BDB0 for ; Thu, 23 Feb 2017 07:27:35 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA29348; Thu, 23 Feb 2017 09:27:31 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1cgnoI-000Ohv-VA; Thu, 23 Feb 2017 09:27:30 +0200 Subject: Re: zfs raidz overhead To: "Eric A. Borisch" , Wiktor Niesiobedzki References: <1b54a2fe35407a95edca1f992fa08a71@norman-vivat.ru> Cc: "freebsd-fs@freebsd.org" , "Eugene M. Zheganin" From: Andriy Gapon Message-ID: Date: Thu, 23 Feb 2017 09:26:09 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2017 07:27:36 -0000 On 23/02/2017 02:49, Eric A. Borisch wrote: > On Wed, Feb 22, 2017 at 3:50 PM, Wiktor Niesiobedzki wrote: > >> I can add to this, that this is not only seen on raidz, but also on >> mirror pools, such as this: >> # zpool status tank >> pool: tank >> state: ONLINE >> scan: scrub repaired 0 in 3h22m with 0 errors on Thu Feb 9 06:47:07 2017 >> config: >> >> NAME STATE READ WRITE CKSUM >> tank ONLINE 0 0 0 >> mirror-0 ONLINE 0 0 0 >> gpt/tank1.eli ONLINE 0 0 0 >> gpt/tank2.eli ONLINE 0 0 0 >> >> errors: No known data errors >> >> >> When I createted test zvols: >> # zfs create -V10gb -o volblocksize=8k tank/tst-8k >> # zfs create -V10gb -o volblocksize=16k tank/tst-16k >> # zfs create -V10gb -o volblocksize=32k tank/tst-32k >> # zfs create -V10gb -o volblocksize=64k tank/tst-64k >> # zfs create -V10gb -o volblocksize=128k tank/tst-128k >> >> # zfs get used tank/tst-8k >> NAME PROPERTY VALUE SOURCE >> tank/tst-8k used 10.3G - >> root@kadlubek:~ # zfs get used tank/tst-16k >> NAME PROPERTY VALUE SOURCE >> tank/tst-16k used 10.2G - >> root@kadlubek:~ # zfs get used tank/tst-32k >> NAME PROPERTY VALUE SOURCE >> tank/tst-32k used 10.1G - >> root@kadlubek:~ # zfs get used tank/tst-64k >> NAME PROPERTY VALUE SOURCE >> tank/tst-64k used 10.0G - >> root@kadlubek:~ # zfs get used tank/tst-128k >> NAME PROPERTY VALUE SOURCE >> tank/tst-128k used 10.0G - > > > Nope, that all looks correct. There is space reserved for metadata > (checksum for example) when you create a zvol. Since checksums are computed > by block, it makes sense that more metadata is required for the 8k > volblocksize. Also, the smaller the data blocks the more indirect blocks are needed to cover the same size. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Fri Feb 24 07:38:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37DB8CEBE4E for ; Fri, 24 Feb 2017 07:38:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1A0AF1784 for ; Fri, 24 Feb 2017 07:38:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1O7c2U7001247 for ; Fri, 24 Feb 2017 07:38:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Fri, 24 Feb 2017 07:38:02 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Feb 2017 07:38:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #28 from Andriy Gapon --- Lev, could you please follow up on what you see with the latest patch? Thank you! --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Feb 24 07:38:27 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7065CEBE95 for ; Fri, 24 Feb 2017 07:38:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D6C16181A for ; Fri, 24 Feb 2017 07:38:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v1O7cRBC001817 for ; Fri, 24 Feb 2017 07:38:27 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Fri, 24 Feb 2017 07:38:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: avg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Feb 2017 07:38:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Open Assignee|freebsd-fs@FreeBSD.org |avg@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Feb 25 19:10:16 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88A02CED035 for ; Sat, 25 Feb 2017 19:10:16 +0000 (UTC) (envelope-from Shiva.Bhanujan@Quorum.net) Received: from mail.quorumlabs.com (mail.quorum.net [64.74.133.216]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.quorumlabs.com", Issuer "mail.quorumlabs.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 79A85671 for ; Sat, 25 Feb 2017 19:10:15 +0000 (UTC) (envelope-from Shiva.Bhanujan@Quorum.net) Received: from QLEXC01.Quorum.local ([fe80::79eb:fde8:ed8e:5648]) by QLEXC01.Quorum.local ([fe80::79eb:fde8:ed8e:5648%14]) with mapi id 14.02.0318.001; Sat, 25 Feb 2017 11:09:08 -0800 From: Shiva Bhanujan To: "freebsd-fs@freebsd.org" Subject: RE: FreeBSD restartable send/receive over WAN Thread-Topic: FreeBSD restartable send/receive over WAN Thread-Index: AdKIga96XkuYo0TvTyGgnDeYS4b6VAARaL2AAbS/t0U= Date: Sat, 25 Feb 2017 19:09:06 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701BDB11492@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701BDB0FD40@QLEXC01.Quorum.local>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [24.6.174.236] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2017 19:10:16 -0000 Hi,=0A= =0A= I just tried restartable send/receive in 10.3 and it works like a charm. I= was wondering if compressed send has made its way into FreeBSD? I checked= 10.3 and 11.0-RELEASE, and I don't see the -c/--compressed option. Any po= inters?=0A= =0A= Regards,=0A= Shiva=0A= =0A= =0A= ________________________________________=0A= From: owner-freebsd-fs@freebsd.org [owner-freebsd-fs@freebsd.org] on behalf= of Adam Nowacki [nowakpl@platinum.linux.pl]=0A= Sent: Thursday, February 16, 2017 10:41 AM=0A= To: freebsd-fs@freebsd.org=0A= Subject: Re: FreeBSD restartable send/receive over WAN=0A= =0A= On 2017-02-16 19:22, Shiva Bhanujan wrote:=0A= > Hello,=0A= >=0A= > I was wondering if restartable send/receive is available in FreeBSD? We'= re running 10.2 and have a requirement of sending and receiving ZFS snapsho= ts over a WAN link. The snapshots could be more than a few terabytes.=0A= >=0A= > Can somebody please give me pointers, and if this feature is or isn't ava= ilable in FreeBSD?=0A= =0A= FreeBSD 10.3 and later.=0A= =0A= _______________________________________________=0A= freebsd-fs@freebsd.org mailing list=0A= https://lists.freebsd.org/mailman/listinfo/freebsd-fs=0A= To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"=0A=