From owner-freebsd-fs@freebsd.org Sun Sep 20 17:09:28 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CBA9FA05AAA for ; Sun, 20 Sep 2015 17:09:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B575B103F for ; Sun, 20 Sep 2015 17:09:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8KH9SpW072397 for ; Sun, 20 Sep 2015 17:09:28 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 203201] on zfs extattr behaviour broken after unlink Date: Sun, 20 Sep 2015 17:09:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2015 17:09:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203201 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |patch -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Sun Sep 20 20:03:54 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6AEECA05C64 for ; Sun, 20 Sep 2015 20:03:54 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: from mail-io0-f176.google.com (mail-io0-f176.google.com [209.85.223.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3BC601346 for ; Sun, 20 Sep 2015 20:03:53 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: by iofb144 with SMTP id b144so102167986iof.1 for ; Sun, 20 Sep 2015 13:03:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:reply-to:subject:references:to:from:organization :message-id:date:user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=3j9+zWIHdVD8Mq0tgt1GUw/b/2P022CNOImyyJIfmtg=; b=nCvmMl0swtM78drKqEA5+9jLSf5khaJuD6Q30/VRkQQI4bSONjB2nJlR9GX2LDnbbz WaKRpgewqQVbSPO+UDUrDZlY1MNu0G1xspSLJkM+UyjmfH3CMJllRRI1BYE4IEzR25BT mCX/AsbqCF2VAZGlCWnhCHpWvrIj1CHVdFxEjb+JJoDtbXbf0Z3WaOxJ+B35uazDQ+2i 4yX8/Oo8O1q7jSdhS9U4jkiKJkWQYWdHNpJpUfgz/Sk4vEZUxevNEC9+9ezxCAX79ZZQ MzqxWkHLisKP1N5YMs2RvqY11BKZ8t6m7uFNE0uEKwnGpEVRp2MClnJgXH0KG9GBaAeu wYDg== X-Gm-Message-State: ALoCoQl33QB3+u+f2Yn29IhjOPI2F8t+vRFBA/CjZM+e4LF9QusBLCaIyZPn1HrQcqMIrd+WkMAJ X-Received: by 10.107.160.82 with SMTP id j79mr24019741ioe.18.1442779427751; Sun, 20 Sep 2015 13:03:47 -0700 (PDT) Received: from [192.168.0.18] ([63.231.252.189]) by smtp.googlemail.com with ESMTPSA id c97sm8627782ioj.41.2015.09.20.13.03.44 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 20 Sep 2015 13:03:47 -0700 (PDT) Reply-To: linda@kateley.com Subject: Re: ZFS cpu requirements, with/out compression and/or dedup References: To: freebsd-fs@freebsd.org From: Linda Kateley Organization: Kateley Company Message-ID: <55FF111A.4040300@kateley.com> Date: Sun, 20 Sep 2015 15:03:38 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2015 20:03:54 -0000 I usually don't worry about CPU much until i start running alot of IO, network or disk. Dedup takes a ton of memory too. Any algorithm for TB's of storage and cpu/ram is usually wrong. CPU and memory are important by data coming in and out.. not data at rest. What kind of data? On 9/19/15 6:56 AM, Sami Halabi wrote: > Hi everyone, > I've been searching the documentation, wiki.. etc. but found no rule of > thumb to CPU requirements to run ZFS fs. for memory there are few rules > according to using dedup or not. > > rules of thumb I concluded are so far: > 1. use compression lz4. > 2. use checksum. > 3. disable atime. > > from what i read the status of dedup is not that clear and seems there are > bugs and better to avoid it? > > so according to 1-3 above what cpu requirements i need? is ATOM cpu like > supermicro c2750/3/5/8 enough to run system of 20TB /40TB with 1-3 above? > if dedup IS enabled would it still work fine? > > Thanks in advance, > Sami halabi > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Sun Sep 20 21:00:48 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9272EA05700 for ; Sun, 20 Sep 2015 21:00:48 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6E32D1600 for ; Sun, 20 Sep 2015 21:00:48 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8KL0mrT053203 for ; Sun, 20 Sep 2015 21:00:48 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201509202100.t8KL0mrT053203@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 Date: Sun, 20 Sep 2015 21:00:48 +0000 Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2015 21:00:48 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f 3 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Sun Sep 20 21:11:57 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A716A05D85 for ; Sun, 20 Sep 2015 21:11:57 +0000 (UTC) (envelope-from quartz@sneakertech.com) Received: from douhisi.pair.com (douhisi.pair.com [209.68.5.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BF541E22 for ; Sun, 20 Sep 2015 21:11:56 +0000 (UTC) (envelope-from quartz@sneakertech.com) Received: from [10.2.2.1] (pool-173-48-121-235.bstnma.fios.verizon.net [173.48.121.235]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by douhisi.pair.com (Postfix) with ESMTPSA id C693F3F6FD for ; Sun, 20 Sep 2015 17:11:49 -0400 (EDT) Message-ID: <55FF2115.8010209@sneakertech.com> Date: Sun, 20 Sep 2015 17:11:49 -0400 From: Quartz MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: ZFS cpu requirements, with/out compression and/or dedup References: <55FF111A.4040300@kateley.com> In-Reply-To: <55FF111A.4040300@kateley.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2015 21:11:57 -0000 > Any algorithm for TB's of storage and cpu/ram is usually wrong. dedup is kind of a special case though, because it has to keep the entire DDT in non-paged ram (assuming you want the machine to be usable). Of course, the rule of thumb is for USED space. 40TB of blank space won't need any ram obviously. From owner-freebsd-fs@freebsd.org Sun Sep 20 21:41:09 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57054A06C0A for ; Sun, 20 Sep 2015 21:41:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 42C861291 for ; Sun, 20 Sep 2015 21:41:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8KLf92j065500 for ; Sun, 20 Sep 2015 21:41:09 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 203133] [nfs] [lor] newnfs/proctree lock order reversal Date: Sun, 20 Sep 2015 21:41:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-BETA2 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rmacklem@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2015 21:41:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203133 --- Comment #1 from Rick Macklem --- *** Bug 203134 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Sun Sep 20 21:41:08 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D998DA06C08 for ; Sun, 20 Sep 2015 21:41:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C57B8128A for ; Sun, 20 Sep 2015 21:41:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8KLf8Ro065490 for ; Sun, 20 Sep 2015 21:41:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 203134] [nfs] [nlm] [lor] newnfs/allproc lock order reversal on NFS mount recovery Date: Sun, 20 Sep 2015 21:41:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-BETA2 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rmacklem@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2015 21:41:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203134 Rick Macklem changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rmacklem@FreeBSD.org Status|New |Closed Resolution|--- |DUPLICATE --- Comment #1 from Rick Macklem --- The info below doesn't seem to show where the locks are acquired in the other order. The NFS client (for NFSv4) and the NLM first lock the NFS vnode and then the "proc" related lock(s). Since I do not believe the NFS subsystem and NLM (not really a part of NFS, but a separate protocol) ever first locks the proc structure and then a vnode, I don't think and deadlock can occur. If someone knows of a way that the generic kernel code could lock an NFS client vnode after acquiring a proc lock, please let me know. I do know that it isn't practical to "fix" these LORs, but I do not believe that they can cause deadlocks. If I find out where harmless LORs are listed, I'll add these. rick ps: Although 203133 isn't the same LOR, the same story applies to both. *** This bug has been marked as a duplicate of bug 203133 *** -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Mon Sep 21 12:50:40 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C58F9D0A49 for ; Mon, 21 Sep 2015 12:50:40 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-wi0-x22c.google.com (mail-wi0-x22c.google.com [IPv6:2a00:1450:400c:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EB36710BE for ; Mon, 21 Sep 2015 12:50:39 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: by wicge5 with SMTP id ge5so114418099wic.0 for ; Mon, 21 Sep 2015 05:50:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=m7z16md8bpDtLiLy+PT7k2/yFze75ov6lBoSYBFxUoM=; b=jc07mhNPtbInT3Ux0YRvuiYMd1amsdCHnJlo9I+G9m2uSpqDxnABsSZoY6809zob1a kAB58nMvyctDZt6VB9awwxhHaRaT5e5757N0BgVBaBecQ26m2Rx9YHnbXd5jPucl6lqm GkUaQUu4BS0Zp8UQ+1sp2MJ6mX0SIhEjLBV2GDRpa0c71ki1UkOlzsKzNe50ZxZuhQDA nO0TNvTS/O60JTJOwQ1hQ1c5U4AtsJcfduheNIIm8QZmU7dJNI7n3BHfVfCTHUmKKgz9 fZ+n+waHT658/Fd0eNJ/ApGgc262Nom5wfmqt5JyM8N8o8MVvTY/YvQeI7XqA0dtI/Il JjKQ== MIME-Version: 1.0 X-Received: by 10.194.171.3 with SMTP id aq3mr23188952wjc.54.1442839838252; Mon, 21 Sep 2015 05:50:38 -0700 (PDT) Received: by 10.28.125.18 with HTTP; Mon, 21 Sep 2015 05:50:38 -0700 (PDT) In-Reply-To: <55FD9A2B.8060207@sneakertech.com> References: <55FD9A2B.8060207@sneakertech.com> Date: Mon, 21 Sep 2015 13:50:38 +0100 Message-ID: Subject: Re: ZFS cpu requirements, with/out compression and/or dedup From: krad To: Quartz Cc: FreeBSD FS Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 12:50:40 -0000 "It's also 'permanent' in the sense that you have to turn it on with the > creation of a dataset and can't disable it without nuking said dataset. " This is completely untrue, there performance issues with dedup are limited to writes only, as it needs to check the DDT table for every write to the file system with dedup enabled. Once the data is on the disk there is no overhead, and in many cases a performance boost as less data on the disk means less head movement and its also more likely to be in any available caches. If the write performance does become an issue you can turn it off on that particular file system. This may cause you to no longer have enough capacity on the pool, but then pools are easily extended. From owner-freebsd-fs@freebsd.org Mon Sep 21 12:58:15 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D88D4A05021 for ; Mon, 21 Sep 2015 12:58:15 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 758CF15DD for ; Mon, 21 Sep 2015 12:58:15 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: by wicgb1 with SMTP id gb1so113589149wic.1 for ; Mon, 21 Sep 2015 05:58:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=7NO9e2RM44KOcLiX21rqyWNjYZdBZI6oqxjsBAQBfe8=; b=w8n0APOicXTiFa+gCfDe33P9Z6X5dJreeGcilwKcrME70IHhVUfuOPZS07Wymt1Wie cufJyTILZT6FvTxR28dN7EMgga3wtmEirbu01N4fQMaEz7ww8WLfxffKiQaglsCveoHo 0NptaWTZ6AAkfXFZ2Av6easq0mGIiLeVwBdBuCFzr1OJAGI+7kvzzINfEv2J2Sv9KfMF yeRrVoWelf1GIElcx4O4wx8X9GvIq2EtfG+uypEpnNt9CypF9MiXkWWHSps/Z213NM17 9yiMVAzJylOX4ACm+OlaPQhTnoLYOqc/pLPtqsloTKYCEUr4KuaYefAZpt8JRCSzcvS+ 0KCg== MIME-Version: 1.0 X-Received: by 10.180.104.40 with SMTP id gb8mr13646195wib.17.1442840293969; Mon, 21 Sep 2015 05:58:13 -0700 (PDT) Received: by 10.28.125.18 with HTTP; Mon, 21 Sep 2015 05:58:13 -0700 (PDT) In-Reply-To: References: Date: Mon, 21 Sep 2015 13:58:13 +0100 Message-ID: Subject: Re: ZFS cpu requirements, with/out compression and/or dedup From: krad To: Sami Halabi Cc: FreeBSD FS Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 12:58:15 -0000 An atom with 2gb of ram in theory is fine to run zfs. What is more important is the workload. If you are going to have 100k iops a second, with lots of synchronous writes then you are going to struggle with an atom, and will need something a lot more beefy. I have a custom built router based on an atom, running zfsonroot freebsd 10.2 backed via an ssd. It runs squid, dns, vpn, dhcp, ip6tunnel, and a small website for about 8 users, and it handles this fine. However there is no massive disk io going on even when we are all heavily browsing. If I tried to use the same hardware for an esx datastor running over iscsi, i would expect it to get a little stressed. On 19 September 2015 at 12:56, Sami Halabi wrote: > Hi everyone, > I've been searching the documentation, wiki.. etc. but found no rule of > thumb to CPU requirements to run ZFS fs. for memory there are few rules > according to using dedup or not. > > rules of thumb I concluded are so far: > 1. use compression lz4. > 2. use checksum. > 3. disable atime. > > from what i read the status of dedup is not that clear and seems there are > bugs and better to avoid it? > > so according to 1-3 above what cpu requirements i need? is ATOM cpu like > supermicro c2750/3/5/8 enough to run system of 20TB /40TB with 1-3 above? > if dedup IS enabled would it still work fine? > > Thanks in advance, > Sami halabi > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Mon Sep 21 13:57:46 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ADA5FA06A23 for ; Mon, 21 Sep 2015 13:57:46 +0000 (UTC) (envelope-from quartz@sneakertech.com) Received: from douhisi.pair.com (douhisi.pair.com [209.68.5.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8DDEF107F for ; Mon, 21 Sep 2015 13:57:46 +0000 (UTC) (envelope-from quartz@sneakertech.com) Received: from [10.2.2.1] (pool-173-48-121-235.bstnma.fios.verizon.net [173.48.121.235]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by douhisi.pair.com (Postfix) with ESMTPSA id 8B4253F7AB for ; Mon, 21 Sep 2015 09:57:44 -0400 (EDT) Message-ID: <56000CD8.4030208@sneakertech.com> Date: Mon, 21 Sep 2015 09:57:44 -0400 From: Quartz MIME-Version: 1.0 To: FreeBSD FS Subject: Re: ZFS cpu requirements, with/out compression and/or dedup References: <55FD9A2B.8060207@sneakertech.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 13:57:46 -0000 > This is completely untrue, there performance issues with dedup are > limited to writes only, as it needs to check the DDT table for every > write to the file system with dedup enabled. Once the data is on the > disk there is no overhead, and in many cases a performance boost as less > data on the disk means less head movement and its also more likely to be > in any available caches. If the write performance does become an issue > you can turn it off on that particular file system. This may cause you > to no longer have enough capacity on the pool, but then pools are easily > extended. It still needs to keep the tables in memory as long as there's still deduped data on disk though, right? Else what keeps track of which blocks are used by which files? From owner-freebsd-fs@freebsd.org Mon Sep 21 14:10:48 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 21853A06FA9 for ; Mon, 21 Sep 2015 14:10:48 +0000 (UTC) (envelope-from quartz@sneakertech.com) Received: from douhisi.pair.com (douhisi.pair.com [209.68.5.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 02610183E for ; Mon, 21 Sep 2015 14:10:47 +0000 (UTC) (envelope-from quartz@sneakertech.com) Received: from [10.2.2.1] (pool-173-48-121-235.bstnma.fios.verizon.net [173.48.121.235]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by douhisi.pair.com (Postfix) with ESMTPSA id C50DD3F7AB for ; Mon, 21 Sep 2015 10:10:46 -0400 (EDT) Message-ID: <56000FE6.3000306@sneakertech.com> Date: Mon, 21 Sep 2015 10:10:46 -0400 From: Quartz MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: ZFS cpu requirements, with/out compression and/or dedup References: <55FF111A.4040300@kateley.com> <55FF2115.8010209@sneakertech.com> In-Reply-To: <55FF2115.8010209@sneakertech.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 14:10:48 -0000 >> Any algorithm for TB's of storage and cpu/ram is usually wrong. > > dedup is kind of a special case though, because it has to keep the > entire DDT in non-paged ram (assuming you want the machine to be usable). > > Of course, the rule of thumb is for USED space. 40TB of blank space > won't need any ram obviously. Also, just for reference: according to the specs each entry in the dedup table costs about 320 bytes of memory per block of disk. This means that AT BEST (assuming ZFS decides to use full 128K blocks in your case) you'll need 2.5GB of ram per 1 TB of used space just for the DDT stuff (not counting ARC and everything else). Most systems are probably not going to be lucky enough to have 128K blocks though, so in real world terms you're looking at several GB of ram per TB of disk, and in worst case scenarios you might need a couple hundred GB... but at that point you should be offloading the DDT onto fast SSD L2ARC. From owner-freebsd-fs@freebsd.org Mon Sep 21 14:11:42 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EFF9A05111 for ; Mon, 21 Sep 2015 14:11:42 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9BF541973 for ; Mon, 21 Sep 2015 14:11:41 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: by wiclk2 with SMTP id lk2so113655268wic.1 for ; Mon, 21 Sep 2015 07:11:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=EexZLTbFDwSHzh1X6fNeqOiVIuNG4qMHevlq/Ux4VI8=; b=rHrdpYQBY7ryOccwH3ji33Atw+W1MuwAnveJyOvTfz7qYTSuAUjkN1lyfLja5aU95R l//3ZrE9OyV0O/Bl3i657GnTikZ2qclkpDLvlyhZI3rSkDIXkkHAxyVT1aQYWgmQxzwH xX4sef+pfKjxBBFWDhr3JRpSSjf7harcTbNagFJCW2CmTmM61qcFAgv4D4nlNUENjNnE q+XnStig2Nm5fjpniMI8G/nBXw/u3Aa+fod7fWwU2RGeUzFL4avU9jdwrLI4coMh83AI x1TTffJwv0Hl+14nzosbMeS3oKBj6VODRGGP0/XhTvmXU0VjM2zepddrswwTrulZIJtc W4bw== MIME-Version: 1.0 X-Received: by 10.181.27.138 with SMTP id jg10mr14648827wid.29.1442844700012; Mon, 21 Sep 2015 07:11:40 -0700 (PDT) Received: by 10.28.125.18 with HTTP; Mon, 21 Sep 2015 07:11:39 -0700 (PDT) In-Reply-To: <56000CD8.4030208@sneakertech.com> References: <55FD9A2B.8060207@sneakertech.com> <56000CD8.4030208@sneakertech.com> Date: Mon, 21 Sep 2015 15:11:39 +0100 Message-ID: Subject: Re: ZFS cpu requirements, with/out compression and/or dedup From: krad To: Quartz Cc: FreeBSD FS Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 14:11:42 -0000 Nope DDT is only used for writes, zfs uses a free block space map, so only when a block is completely unreferenced will it be written to. The DDT is a table of blocks and their checksums. https://blogs.oracle.com/bonwick/en/entry/space_maps http://www.c0t0d0s0.org/archives/7271-ZFS-Dedup-Internals.html there are probably better references On 21 September 2015 at 14:57, Quartz wrote: > This is completely untrue, there performance issues with dedup are >> limited to writes only, as it needs to check the DDT table for every >> write to the file system with dedup enabled. Once the data is on the >> disk there is no overhead, and in many cases a performance boost as less >> data on the disk means less head movement and its also more likely to be >> in any available caches. If the write performance does become an issue >> you can turn it off on that particular file system. This may cause you >> to no longer have enough capacity on the pool, but then pools are easily >> extended. >> > > It still needs to keep the tables in memory as long as there's still > deduped data on disk though, right? Else what keeps track of which blocks > are used by which files? > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Mon Sep 21 17:10:03 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C612A06706 for ; Mon, 21 Sep 2015 17:10:03 +0000 (UTC) (envelope-from marcus@odin.blazingdot.com) Received: from odin.blazingdot.com (odin.blazingdot.com [204.109.60.170]) by mx1.freebsd.org (Postfix) with ESMTP id B558D148A for ; Mon, 21 Sep 2015 17:10:02 +0000 (UTC) (envelope-from marcus@odin.blazingdot.com) Received: by odin.blazingdot.com (Postfix, from userid 1001) id 602031324CA; Mon, 21 Sep 2015 10:02:16 -0700 (PDT) Date: Mon, 21 Sep 2015 10:02:16 -0700 From: Marcus Reid To: Bob Friesenhahn Cc: Sami Halabi , freebsd-fs@freebsd.org Subject: Re: ZFS cpu requirements, with/out compression and/or dedup Message-ID: <20150921170216.GA98888@blazingdot.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Coffee-Level: nearly-fatal User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 17:10:03 -0000 On Sat, Sep 19, 2015 at 09:04:48AM -0500, Bob Friesenhahn wrote: > CPU usage is rarely a problem for zfs except for when compression is > involved. Lz4 is supposed to be CPU-efficient on Intel CPUs. This is misleading. lz4 compression is so fast that in the common case it _increases_ performance. This seems unintuitive until you realize that the time it takes to compress the data is much smaller than the time it takes to write the data to the media, so when you write, say, 25% less data to disk, that translates to time saved. In addition, lz4 has early-abort where it will detect that the data is uncompressible, and just write it out when it is instead of compressing it. This makes it so that you only "pay for" compression when it actually does something. It works really well, and that's why it's enabled by default. Marcus From owner-freebsd-fs@freebsd.org Mon Sep 21 17:48:48 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53D49A05CF4 for ; Mon, 21 Sep 2015 17:48:48 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 167531A5A for ; Mon, 21 Sep 2015 17:48:48 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [127.0.0.1] (unknown [89.113.128.32]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id 077EB2CB7 for ; Mon, 21 Sep 2015 20:48:45 +0300 (MSK) Reply-To: lev@FreeBSD.org To: freebsd-fs@freebsd.org From: Lev Serebryakov Subject: ZFS vs torrents (transmission-bt) X-Enigmail-Draft-Status: N1110 Organization: FreeBSD Message-ID: <560042FD.8080903@FreeBSD.org> Date: Mon, 21 Sep 2015 20:48:45 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 17:48:48 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I've noticed, than when transmission downloads multi-file torrent to ZFS, some files are not shown in "ls" output till they are closed (completed). But not always! Some incomplete files are shown. I have "preallocation" option turned on and partially-downloaded files are named with ".part" extension and renamed to true name after completion. Some such files were lost when transmission-daemon was killed by OOM. I've never seen such behavior before, when my NAS was configured with UFS2. What it could be?! - -- // Lev Serebryakov -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJWAEL8XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGOTZEMUNBMEI1RjQzMThCNjc0QjMzMEFF QUIwM0M1OEJGREM0NzhGAAoJEOqwPFi/3EePxm8P/RjHrN+SegbsIMDmUP9Nt3kC iSZBzf2YelA1T+jVhuCMiTNu7BjyLbNLYwNb1J1/AUOAQzGEb5IR6owi2RlPWqla iyEzzbQSR2Nm2hCHlbIA+ukLER8zoQxReouqb16qFmW+xoVcUnAAWBEwmLA3N28n xFy/ShTteSj20oe+wLowyA8pPv0KH/f8bazz+XkTdvdXL6gi6HnaBd7OjY02zJaU P6nzMHUcofdkg7KfDvoxjydO1qQ+oUyIg69da+NglZVpAvEEDo+XOTy8uijKCjqy rYcdyKZeLLzSfEDMkoHAAudONYv9jg+0t+IhC5BvO1ZracZy8LbRKmW+MlmAre8H 1qZBawbYP947AIqkECPbIfc/jMCfaOKDFj6JHfI7ar3UDrwXVnbW5xCRKYVFIqCS W7UqPKgsRUZs5ujhG0CNpuSw9G9CSOSnTakSPpSHBBrSaC11gSQ/HEXW6LZ2VKTY NjAEAnSQ3oUy7tKujuiABQOZD0p+Cqef3P7ZsVKkZvK38LPaMdnb9PHzfCoOiX2y INP7oYONBHGndp4jlqJRl6/QZgnzN3c5NT95WenByfbuivQbOrjZb5Pk/Gaxqhx8 TPdqHUHfyX7y9oT/aR3s+EX/4qnr2wT9Ih8FeUY2Otirpvz2+3gZJaDcHFI+93wm G/SnjFScvZFw/tb2jUN3 =8kx9 -----END PGP SIGNATURE----- From owner-freebsd-fs@freebsd.org Mon Sep 21 18:38:24 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25326A05B26 for ; Mon, 21 Sep 2015 18:38:24 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from mango.stankevitz.com (mango.stankevitz.com [208.79.93.194]) by mx1.freebsd.org (Postfix) with ESMTP id 14E211D2E for ; Mon, 21 Sep 2015 18:38:23 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from Chriss-MacBook-Pro.local (209-203-101-124.static.twtelecom.net [209.203.101.124]) (using TLSv1.2 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mango.stankevitz.com (Postfix) with ESMTPSA id D9F401072 for ; Mon, 21 Sep 2015 11:28:47 -0700 (PDT) Message-ID: <56004C68.4020904@stankevitz.com> Date: Mon, 21 Sep 2015 11:28:56 -0700 From: Chris Stankevitz User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: FreeBSD Filesystems Subject: Name/label/id metadata: how do I make it go away Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 18:38:24 -0000 Hi, For those wanting to teach me to fish: In the context of naming a disk that has no partitions, can you explain to me the difference among these terms: - gpt label - gpt id - glabel - gpart - /dev/diskid/* - disk_ident - geom - bsdlabel For those wanting to give me a fish: I have a zfs pool of "entire disks". "zpool status" shows some disks with their daX name (which I prefer) and some with a hideously ugly name such as DISK-%20%20%20%20%20-WD-WMC4NOH1ASDF I can make the hideously ugly names temporarily go away with: zpool export foo glabel stop diskid/DISK-%20%20%20%20%20-WD-WMC4NOH1ASDF zpool import foo However I cannot make them permanently go away: zpool export foo glabel clear da6 > Can't clear metadata on da6: Invalid argument. In a prior life these drives were in a FreeNAS setup which presumably added the labels. I did not erase the labels before I recommissioned the drives in my new FreeBSD setup. I'm happy to use dd to erase the metadata but I do not know how the labels were added. Presumably when I understand that I will know which blocks need to be zero-ed out. I need to "surgically" dd to avoid data loss from my zfs disks. I have backups.... actually the problem pool is a backup pool. Thank you, Chris From owner-freebsd-fs@freebsd.org Mon Sep 21 18:58:44 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 569A7A069A3 for ; Mon, 21 Sep 2015 18:58:44 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-oi0-x236.google.com (mail-oi0-x236.google.com [IPv6:2607:f8b0:4003:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1E17419BD for ; Mon, 21 Sep 2015 18:58:44 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by oiww128 with SMTP id w128so63650922oiw.2 for ; Mon, 21 Sep 2015 11:58:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=V+sXNzSrJnUlwzi61SL2yND6jiuxmtUd8F8SKhPB/C0=; b=zcXngnp/17BoQeE7LcKjQEYl6TDrAlZGWy4mACk2nMPNeiAVdRQXbtL1szo+7ieLak CNauH+RiXZXcoASio41thlfO/EDzyDtUmVn2rqnQxYxbGVv0WbI9yGFi55d6vC+XwtGP GxQAy1dR0BXhHub65FXKmCpCHlqkuRRXCjUkWiwIwek7QVgz1+1z+BwZKrc9HWSSzy3A RkFvfyUhi8hjGQ7YHKNiVGS/b+XmVqUYwi5BcUBN6w6/Z+49AjgXNyDtK1EqgDOiScuK HKaSUSZSyUbHQJE9NWZZv1Qlx8qnPmsZiv3Wue8hLCP6pRpxASDQqxauCUzb/gYWYhA7 GFFw== MIME-Version: 1.0 X-Received: by 10.202.228.210 with SMTP id b201mr9684586oih.100.1442861923374; Mon, 21 Sep 2015 11:58:43 -0700 (PDT) Received: by 10.76.81.100 with HTTP; Mon, 21 Sep 2015 11:58:43 -0700 (PDT) In-Reply-To: <56004C68.4020904@stankevitz.com> References: <56004C68.4020904@stankevitz.com> Date: Mon, 21 Sep 2015 11:58:43 -0700 Message-ID: Subject: Re: Name/label/id metadata: how do I make it go away From: Freddie Cash To: Chris Stankevitz Cc: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 18:58:44 -0000 On Mon, Sep 21, 2015 at 11:28 AM, Chris Stankevitz wrote: > Hi, > > > For those wanting to teach me to fish: > > In the context of naming a disk that has no partitions, can you explain t= o > me the difference among these terms: > - gpt label > - gpt id > - glabel > - gpart > - /dev/diskid/* > - disk_ident > - geom > - bsdlabel > > For those wanting to give me a fish: > > I have a zfs pool of "entire disks". "zpool status" shows some disks wit= h > their daX name (which I prefer) and some with a hideously ugly name such = as > DISK-%20%20%20%20%20-WD-WMC4NOH1ASDF > > I can make the hideously ugly names temporarily go away with: > > zpool export foo > glabel stop diskid/DISK-%20%20%20%20%20-WD-WMC4NOH1ASDF > zpool import foo > > However I cannot make them permanently go away: > zpool export foo > glabel clear da6 > > Can't clear metadata on da6: Invalid argument. > > In a prior life these drives were in a FreeNAS setup which presumably > added the labels. I did not erase the labels before I recommissioned the > drives in my new FreeBSD setup. I'm happy to use dd to erase the metadat= a > but I do not know how the labels were added. Presumably when I understan= d > that I will know which blocks need to be zero-ed out. I need to > "surgically" dd to avoid data loss from my zfs disks. I have backups.... > actually the problem pool is a backup pool. > > Thank you, > > Chris > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > =E2=80=8BAdd the following to /boot/loader.conf to make the diskid/* labels disappear forever: =E2=80=8Bkern.geom.label.gptid.enable=3D"0" # Disable the auto-generated GP= T UUIDs for disks kern.geom.label.ufsid.enable=3D"0" # Disable the auto-generated UFS UUIDs f= or filesystems --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@freebsd.org Mon Sep 21 21:10:21 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 100F4A06F0F for ; Mon, 21 Sep 2015 21:10:21 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B12261777 for ; Mon, 21 Sep 2015 21:10:20 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by wiclk2 with SMTP id lk2so130215708wic.1 for ; Mon, 21 Sep 2015 14:10:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:message-id:date :to:mime-version; bh=KAEBsAQRCo9sP4NTUR6U207GT+4Kzq+LRDIcFFoIVBw=; b=m1+wB2I6Ik35H9c6ArgsVtDWqORowBeL0Ab8RcuEmNZEGoPgOkmPZp3gCubjcOKXD/ qYLqiU5wDKaKs9TNxb1JfnXrmVyKgOXsQ6A4UIHHmgbKvWBGUwkddgUrIwsD/Dyh3SZK 3/xA33PhWKJmCjPXAH5ZH7KyN6LPfXKZ3UGr/LkF+DwMTyqbYtRQU5ujrvDG6O18i7Mp po18qVZD+FCGINZP4I+UUrvnymBiYtVyI05WlQkGs9sxMcW+WZMLCVM1rX4ZK4s48VyY OPMGLBV5D3hy8fD1gsPkb6yslRIS4V8BXm5u3BszcmG9dqG4HP/B+zhR11FSNUY2W9hy uXSw== X-Received: by 10.180.186.10 with SMTP id fg10mr16005023wic.30.1442869818206; Mon, 21 Sep 2015 14:10:18 -0700 (PDT) Received: from [192.168.0.139] (cag06-2-82-237-68-117.fbx.proxad.net. [82.237.68.117]) by smtp.gmail.com with ESMTPSA id pl7sm15401297wic.4.2015.09.21.14.10.17 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 21 Sep 2015 14:10:17 -0700 (PDT) From: Ben RUBSON Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: [HAST] ZFS, many disks, write order Message-Id: Date: Mon, 21 Sep 2015 23:10:15 +0200 To: freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 21:10:21 -0000 Hello, I plan to use HAST to synchronize a ZFS pool between 2 servers. The ZFS pool has 3 RAID-Z2 VDEVs (8+2), + 3 spares. +1 mirror for SLOG. +1 mirror for L2ARC. So a total of 37 disks (4TB each). 40Gb/s network bandwidth between the 2 servers. Will I have to define each of the 37 disks/resources in hast.conf ? Stupid question, but will this setup (with so many resources) work ? Will write IOs be ordered on the secondary node in the same order (over = all the 37 devices) as they occurred on the primary node ? This of course to have the secondary node consistent, even after a power = failure of the primary during a high IO load, leading into an import -F = on the secondary node. In HAST, each resource seems to be "independent" from the others. In DRBD, as an example, we can put several volumes in a same resource to = guarantee write order over all the volumes (disks) of the resource. Example : resource r0 { volume 0 { device /dev/drbd0; disk /dev/c0v0; } volume 1 { device /dev/drbd1; disk /dev/c0v1; } } What about HAST then ? Of course I would have liked to have my 37 disks as volumes in the same = resource, as in the example above. Thank you very much for your help ! Best regards, Ben From owner-freebsd-fs@freebsd.org Mon Sep 21 21:13:52 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9EB9A05085 for ; Mon, 21 Sep 2015 21:13:52 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (vps.rulingia.com [103.243.244.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps.rulingia.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5AEAE1B09 for ; Mon, 21 Sep 2015 21:13:51 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (c220-239-242-83.belrs5.nsw.optusnet.com.au [220.239.242.83]) by vps.rulingia.com (8.15.2/8.15.2) with ESMTPS id t8LLDfFq058155 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Tue, 22 Sep 2015 07:13:46 +1000 (AEST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id t8LLDZ2a090441 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 22 Sep 2015 07:13:35 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id t8LLDZXK090440 for freebsd-fs@freebsd.org; Tue, 22 Sep 2015 07:13:35 +1000 (AEST) (envelope-from peter) Date: Tue, 22 Sep 2015 07:13:35 +1000 From: Peter Jeremy To: freebsd-fs@freebsd.org Subject: Re: ZFS cpu requirements, with/out compression and/or dedup Message-ID: <20150921211335.GB41102@server.rulingia.com> References: <20150921170216.GA98888@blazingdot.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="8P1HSweYDcXXzwPJ" Content-Disposition: inline In-Reply-To: <20150921170216.GA98888@blazingdot.com> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender succeeded STARTTLS authentication, not delayed by milter-greylist-4.4.3 (vps.rulingia.com [103.243.244.15]); Tue, 22 Sep 2015 07:13:46 +1000 (AEST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 21:13:52 -0000 --8P1HSweYDcXXzwPJ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2015-Sep-21 13:50:38 +0100, krad wrote: >"It's also 'permanent' in the sense that you have to turn it on with the >> creation of a dataset and can't disable it without nuking said dataset. " > >This is completely untrue, there performance issues with dedup are limited >to writes only, as it needs to check the DDT table for every write to the >file system with dedup enabled. Well, it's partially true. Once you enable dedup on a dataset, it creates a DDT and it's not possible to remove the DDT without nuking the dataset. There are basically 3 operations on a block: Read a block: DDT is never referenced. Write a new block: DDT is referenced is dedup is enabled. Free a block: DDT is always referenced if it exists. The usual "fall off a cliff" scenario is when you go to delete a large file or snapshot on a dataset where dedup has been enabled at some point in the past, even if it's not enabled now. Every block in that file or snapshot is checked against the DDT. Since the DDT is basically a very large hash table this entails lots of random I/O. On 2015-Sep-21 10:10:46 -0400, Quartz wrote: >Also, just for reference: according to the specs each entry in the dedup >table costs about 320 bytes of memory per block of disk. This means that >AT BEST (assuming ZFS decides to use full 128K blocks in your case) >you'll need 2.5GB of ram per 1 TB of used space just for the DDT stuff And at worst, assuming advanced format disks, you'll have 4K blocks and need 80GB RAM per 1 TB used space. In general, the downsides of dedup outweigh the benefits. If you already have the data in ZFS, you can use 'zdb -S' to see what effect rebuilding the pool with dedup enabled would have - how much disk space you will save and how big the DDT is (and hence how much RAM you will need). If you can afford it, make sure you keep good backups, enable DDT and be ready to nuke the pool and restore from backups if dedup doesn't work out. On 2015-Sep-21 10:02:16 -0700, Marcus Reid wrote: >This is misleading. lz4 compression is so fast that in the common case >it _increases_ performance. This is true of most of the compression algorithms. >In addition, lz4 has early-abort where it will detect that the data is >uncompressible, and just write it out when it is instead of compressing >it. I'm not sure how lz4 decides data is uncompressible without trying to compress it. The way ZFS compression works is that it tries to compress a block. Unless the compressed data is small enough to fit into a smaller block (ie 2:1 compression or better), the uncompressed data is stored. (And blocks of NULs are "stored" as holes it the file without attempting compression). In general, unless you know a dataset will always be filled with pre-compressed data - videos, non-RAW images, distfiles - you are better off enabling compression. --=20 Peter Jeremy --8P1HSweYDcXXzwPJ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJWAHL/XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFRUIyOTg2QzMwNjcxRTc0RTY1QzIyN0Ux NkE1OTdBMEU0QTIwQjM0AAoJEBall6Dkogs0WfkQAJkJBxN2A/+yhx8A8T3sF3vz +4cX4k7ThFo68NAK9qwLRcj2V/dnPq+PgR9iIrhp4G1mwkTynoFJe78dCrND6rRU rPIKt9sQOWKm0TqLaVCX2xGfmG/DZquh4d8WmkMqbWycoAScnUfkLXEEn1rP0EVS wc0V4emmkD/AIYZ9zjGAfV9mHIn1p+uyVMdB2ATnA6e2WdkfnFImIc1tzYjF32cc bkI1NPEe89wKPfTl0PTl1hO2Aubs5xFlSpqLXPfB4BqpnKdH2d0ZRbBVb7Gmjrd8 G4sEtMfkqr5Yx7EI9bsbowZWtja+Cb7OWgs3+0DV/xeODwZunKQeJm4SUORQkvFE Pwjv3+gWptwiSKneeqwZXAgME9S0pPUY364PIfwBijTmu7pzYqNbkAHYuvc1e3vz o1h67mE91dW4r4vIGSbUDTI0Gd1mFWsJHyHrXdrYa8+FZXUnixOOSqyAsLTUr0bp q/6M+/BevmE5hLRejqFmTNjEQ8KA5xRiSofwu2TTsp1vEaEgWx5n47IAzIrPtJ6B 878wfVSyOw0S/K6br2LJONKTc/G9lJN/RAFMrxuUP2IPHJT+A/4utCKRg5Ytyf/W e6DP6IotydMJUnGRJSnw1iccu7PNGFbxLXb4Eo1F2tbOjymZiaBq6tgMKBMQ+zvw G+b15asyXYbQhSjGL/vy =DFQV -----END PGP SIGNATURE----- --8P1HSweYDcXXzwPJ-- From owner-freebsd-fs@freebsd.org Mon Sep 21 21:41:50 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EC054A05F2E for ; Mon, 21 Sep 2015 21:41:50 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 924A31CA1 for ; Mon, 21 Sep 2015 21:41:50 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from liminal.local (liminal.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3636:3bff:fed4:b0d6]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.15.2/8.15.2) with ESMTPSA id t8LLfgBv059569 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 21 Sep 2015 22:41:43 +0100 (BST) (envelope-from matthew@FreeBSD.org) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org DKIM-Filter: OpenDKIM Filter v2.10.3 smtp.infracaninophile.co.uk t8LLfgBv059569 Authentication-Results: smtp.infracaninophile.co.uk/t8LLfgBv059569; dkim=none; dkim-atps=neutral X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host liminal.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3636:3bff:fed4:b0d6] claimed to be liminal.local Subject: Re: ZFS cpu requirements, with/out compression and/or dedup To: freebsd-fs@freebsd.org References: <20150921170216.GA98888@blazingdot.com> <20150921211335.GB41102@server.rulingia.com> From: Matthew Seaman X-Enigmail-Draft-Status: N1110 Message-ID: <5600798B.3010208@FreeBSD.org> Date: Mon, 21 Sep 2015 22:41:31 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <20150921211335.GB41102@server.rulingia.com> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="O2MrgKL7ShGv9ETuuIKSiFgd0NsfHJUMP" X-Virus-Scanned: clamav-milter 0.98.7 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 21:41:51 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --O2MrgKL7ShGv9ETuuIKSiFgd0NsfHJUMP Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 21/09/2015 22:13, Peter Jeremy wrote: > In general, the downsides of dedup outweigh the benefits. If you alrea= dy > have the data in ZFS, you can use 'zdb -S' to see what effect rebuildin= g > the pool with dedup enabled would have - how much disk space you will s= ave > and how big the DDT is (and hence how much RAM you will need). If you = can > afford it, make sure you keep good backups, enable DDT and be ready to = nuke > the pool and restore from backups if dedup doesn't work out. Nuking the entire pool is a little heavy handed. Dedup can be turned on and off on a per-ZFS basis. If you've a ZFS that had dedup enabled, you can remove the effects by zfs send / zfs recv to create a pristine un-deduped copy of the data, destroy the original zfs and rename the new one to take its place. Of course, this depends on your having enough free space in the pool to be able to duplicate (and then some) that ZFS. Failing that, you might be able to 'zpool split' if your pool is composed entirely of mirrors. So long as you're able to do without resilience for a while this basically doubles the space you have available to play with. You can then destroy the contents of one of the split zpools, and zfs send the data over from the other split pool. Unfortunately there isn't a reciprocal 'zfs rejoin' command that undoes the splitting, so you'll have to destroy one of the splits and re-add the constituent devices back to restore the mirroring in the other split. Which is a delicate operations and not one which is forgiving of mistakes. And failing that, you can start pushing data over the network, but that's hardly different to restoring from backup. However, either of the first two choices should be significantly faster if you have large quantities of data to handle. Cheers, Matthew --O2MrgKL7ShGv9ETuuIKSiFgd0NsfHJUMP Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2 iQJ8BAEBCgBmBQJWAHmSXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkATd+wP/2PQGIgUJYI8tn+ygFbe0AHO IL2QcwFy2cE/CTz8ubkqQm6zAvTZ//EYez2tQV+tpf/IWk6kRa194I2fsFG7D0/o Fa9N7P0AQ11Skv5BRI6bjjIPFHfvazg0aDdiEQH5iC4AsJMYT0J9+/fqaY8idjnX VG921iKSbl9EXtRKBJhNz8uJyvMI1Zxl0DorKce+lrpGqHNBIjBLs8jwGPlQ3Fm4 yKGBHPUOrQtT0JDQZzDyCWl7ed1TR4OG3pZRvLIHKTz1tEoWGfB69e25eMsnXXSt 2kGbhbKWHndl3xre7yj1lqxsoqyfBuMmke9TLcRuVAhbYVzOuEeQQAozpWYgfKnJ X9VjvRFN+n9sLhUF9iWmdGNF9GkgIBMPh5XzwDIypuZCAYiZuGn7Dx6QojG8dQSi LQDfA7T1On/m6uFtOn11xhSI5Fg4lytlZtwD+8UWAYenOcM1USzVoyrOayWB+Xvh 6B5kFJMENEvxXPYpwTchoUo5oAyl6T687ytJvyu5/a1rhF/uANF6u1ljP4y4gPnb Glw2FRiWZpWAWrkGYXroxmMvPRtJ97S95D5vDUqC3erah3wchFW/51KU28UVxPM/ x9tyeHmXMLCmQYagecEClnSDBixfGIh1eAR0In4sNqWoANiyLd3uuVn1EYWngMaX 5a8w3Q+oKHI4XlujSDyU =PWfM -----END PGP SIGNATURE----- --O2MrgKL7ShGv9ETuuIKSiFgd0NsfHJUMP-- From owner-freebsd-fs@freebsd.org Tue Sep 22 03:41:31 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0647A0261B for ; Tue, 22 Sep 2015 03:41:31 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B4B251D16 for ; Tue, 22 Sep 2015 03:41:31 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.15.2/8.15.2) with ESMTPS id t8M3fTWb080542 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 21 Sep 2015 21:41:29 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.15.2/8.15.2/Submit) with ESMTP id t8M3fTji080539; Mon, 21 Sep 2015 21:41:29 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Mon, 21 Sep 2015 21:41:28 -0600 (MDT) From: Warren Block To: Chris Stankevitz cc: FreeBSD Filesystems Subject: Re: Name/label/id metadata: how do I make it go away In-Reply-To: <56004C68.4020904@stankevitz.com> Message-ID: References: <56004C68.4020904@stankevitz.com> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Mon, 21 Sep 2015 21:41:29 -0600 (MDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 03:41:32 -0000 On Mon, 21 Sep 2015, Chris Stankevitz wrote: > For those wanting to teach me to fish: > > In the context of naming a disk that has no partitions, can you explain to me > the difference among these terms: > - gpt label A manually-assigned label in the GPT metadata. Requires GPT partitioning. > - gpt id A system-assigned ID in the GPT metadata. Requires GPT partitioning. > - glabel A pgoram that can manually assigns labels to arbitrary devices. The last block of the device is used for metadata, so the created device is one block smaller than the raw device. See glabel(8). > - gpart FreeBSD's versatile disk partitioning program. See gpart(8). > - /dev/diskid/* > - disk_ident A GUID assigned to the disk? I forget. > - geom FreeBSD's GEOM system that allows adding filters or functions to bare disk devices, or the program to manipulate it. See geom(4) and geom(8). > - bsdlabel Both an obsolete disk partitioning format and the program to create it. Invented before MBR, then usually used in an awful combination with MBR to make up for MBR's weaknesses. See bsdlabel(8). gpart(8) can create these and other formats, but most of the time GPT is the most powerful and easiest to use. > For those wanting to give me a fish: > > I have a zfs pool of "entire disks". "zpool status" shows some disks with > their daX name (which I prefer) and some with a hideously ugly name such as > DISK-%20%20%20%20%20-WD-WMC4NOH1ASDF Disable those labels with kern.geom.label.gptid.enable="0" in /boot/loader.conf. From owner-freebsd-fs@freebsd.org Tue Sep 22 06:10:41 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CAF6A020A4 for ; Tue, 22 Sep 2015 06:10:41 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from mango.stankevitz.com (mango.stankevitz.com [208.79.93.194]) by mx1.freebsd.org (Postfix) with ESMTP id 7858D1A12 for ; Tue, 22 Sep 2015 06:10:41 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from Chriss-MacBook-Pro.local (209-203-101-124.static.twtelecom.net [209.203.101.124]) (using TLSv1.2 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mango.stankevitz.com (Postfix) with ESMTPSA id 7692610E4; Mon, 21 Sep 2015 23:10:40 -0700 (PDT) Subject: Re: Name/label/id metadata: how do I make it go away To: Warren Block References: <56004C68.4020904@stankevitz.com> Cc: FreeBSD Filesystems From: Chris Stankevitz Message-ID: <5600F0DF.8000805@stankevitz.com> Date: Mon, 21 Sep 2015 23:10:39 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 06:10:41 -0000 Warren/Freddie, Thank you for your replies. On 9/21/15 8:41 PM, Warren Block wrote: >> - gpt id > > A system-assigned ID in the GPT metadata. Requires GPT partitioning. >> I have a zfs pool of "entire disks". "zpool status" shows some disks >> with their daX name (which I prefer) and some with a hideously ugly >> name such as DISK-%20%20%20%20%20-WD-WMC4NOH1ASDF > > Disable those labels with kern.geom.label.gptid.enable="0" in > /boot/loader.conf. Combining your two statements quoted above, I believe I can conclude that my ZFS "whole disk" drives must have some remnants of GPT left over from their previous lives (namely the system-assigned ID in the GPT metadata). Surprisingly, these apparently GPT-supplied labels appear to "go away" when I issue a "glabel stop". I would not expect this given that Warren explained that glabels (whose metadata are stored at the end of the device and completely outside the virtual device) are not the same as GPT labels (whose metadata are stored within the device on the GPT metadata). I believe one of the following must be true: 1. It is possible to use "glabel stop" to disable a "GPT system-assigned ID" -- even though glabel is a tool for manipulating another style of labels. 2. "glabel stop" only affects glabels. In my case my drives must contain "glables" and not "GPT system-assigned IDs" as Warren guessed. 3. I misunderstood and glabels/GPS system-assigned IDs are really the same thing. Thank you again, Chris From owner-freebsd-fs@freebsd.org Tue Sep 22 07:37:14 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15E0EA068A5 for ; Tue, 22 Sep 2015 07:37:14 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id CBDE21CCD for ; Tue, 22 Sep 2015 07:37:13 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id BBA341472004 for ; Tue, 22 Sep 2015 09:27:07 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NbyCFpadUjWz for ; Tue, 22 Sep 2015 09:27:05 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 3E694147200C for ; Tue, 22 Sep 2015 09:27:04 +0200 (CEST) Subject: Re: [HAST] ZFS, many disks, write order References: To: freebsd-fs@freebsd.org Reply-To: jg@internetx.com From: InterNetX - Juergen Gotteswinter Message-ID: <560102BD.6040703@internetx.com> Date: Tue, 22 Sep 2015 09:26:53 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 07:37:14 -0000 One of the great Hast Features is that it Hides Disk Failures from ZFS, this has been already discussed on this List a few Months ago. Single ZIL Flash Drive is also not such a great Idea. I whould suggest to start from scratch with "ZFS Best Practices". Am 21.09.2015 um 23:10 schrieb Ben RUBSON: > Hello, > > I plan to use HAST to synchronize a ZFS pool between 2 servers. > The ZFS pool has 3 RAID-Z2 VDEVs (8+2), + 3 spares. > +1 mirror for SLOG. > +1 mirror for L2ARC. > So a total of 37 disks (4TB each). > 40Gb/s network bandwidth between the 2 servers. > > Will I have to define each of the 37 disks/resources in hast.conf ? > Stupid question, but will this setup (with so many resources) work ? > > Will write IOs be ordered on the secondary node in the same order (over all the 37 devices) as they occurred on the primary node ? > This of course to have the secondary node consistent, even after a power failure of the primary during a high IO load, leading into an import -F on the secondary node. > > In HAST, each resource seems to be "independent" from the others. > In DRBD, as an example, we can put several volumes in a same resource to guarantee write order over all the volumes (disks) of the resource. > Example : > > resource r0 { > volume 0 { > device /dev/drbd0; > disk /dev/c0v0; > } > volume 1 { > device /dev/drbd1; > disk /dev/c0v1; > } > } > > What about HAST then ? > Of course I would have liked to have my 37 disks as volumes in the same resource, as in the example above. > > Thank you very much for your help ! > > Best regards, > > Ben > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Tue Sep 22 07:42:43 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28CDEA06CBC for ; Tue, 22 Sep 2015 07:42:43 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD1BF1171 for ; Tue, 22 Sep 2015 07:42:42 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id A07441472009 for ; Tue, 22 Sep 2015 09:42:40 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3OZ3oHoc1Y+5 for ; Tue, 22 Sep 2015 09:42:38 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 5D3281472004 for ; Tue, 22 Sep 2015 09:42:38 +0200 (CEST) Reply-To: jg@internetx.com, jg@internetx.com Subject: Re: [HAST] ZFS, many disks, write order References: <560102BD.6040703@internetx.com> To: freebsd-fs@freebsd.org From: InterNetX - Juergen Gotteswinter Message-ID: <56010663.3050201@internetx.com> Date: Tue, 22 Sep 2015 09:42:27 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <560102BD.6040703@internetx.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 07:42:43 -0000 my fault, forget my comment about single zil Am 22.09.2015 um 09:26 schrieb InterNetX - Juergen Gotteswinter: > One of the great Hast Features is that it Hides Disk Failures from ZFS, > this has been already discussed on this List a few Months ago. > > Single ZIL Flash Drive is also not such a great Idea. I whould suggest > to start from scratch with "ZFS Best Practices". > > Am 21.09.2015 um 23:10 schrieb Ben RUBSON: >> Hello, >> >> I plan to use HAST to synchronize a ZFS pool between 2 servers. >> The ZFS pool has 3 RAID-Z2 VDEVs (8+2), + 3 spares. >> +1 mirror for SLOG. >> +1 mirror for L2ARC. >> So a total of 37 disks (4TB each). >> 40Gb/s network bandwidth between the 2 servers. >> >> Will I have to define each of the 37 disks/resources in hast.conf ? >> Stupid question, but will this setup (with so many resources) work ? >> >> Will write IOs be ordered on the secondary node in the same order (over all the 37 devices) as they occurred on the primary node ? >> This of course to have the secondary node consistent, even after a power failure of the primary during a high IO load, leading into an import -F on the secondary node. >> >> In HAST, each resource seems to be "independent" from the others. >> In DRBD, as an example, we can put several volumes in a same resource to guarantee write order over all the volumes (disks) of the resource. >> Example : >> >> resource r0 { >> volume 0 { >> device /dev/drbd0; >> disk /dev/c0v0; >> } >> volume 1 { >> device /dev/drbd1; >> disk /dev/c0v1; >> } >> } >> >> What about HAST then ? >> Of course I would have liked to have my 37 disks as volumes in the same resource, as in the example above. >> >> Thank you very much for your help ! >> >> Best regards, >> >> Ben >> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Tue Sep 22 08:17:37 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6CE9A031CA for ; Tue, 22 Sep 2015 08:17:37 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-outbound.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D5101DE4 for ; Tue, 22 Sep 2015 08:17:36 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) by smtp-outbound.userve.net (8.15.1/8.15.1) with ESMTPS id t8M8FBs3037705 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 22 Sep 2015 09:15:12 +0100 (BST) (envelope-from matt.churchyard@userve.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1442909714; bh=vBy05hri4E3H0WGcqBHW0wP2UvtJySdX/XRS8IQ9nNo=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=PTB353bcv+88aLKPyCw8oHw51bauONJZU/TLFHkpHS3Ezk1kWwRMOuiz6zGVmGY8g vt2jjq0JhltUfOcnBsPVP4JWa/6poDhXem2Vlvz4vSlSmY/R6K1rpSNJR27QyuImZp 27+9hPoZ3bZ+ltLs8hez+P2TKYMJolEJKaoPQ11Y= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 22 Sep 2015 09:15:06 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Tue, 22 Sep 2015 09:15:06 +0100 From: Matt Churchyard To: Chris Stankevitz CC: FreeBSD FS Subject: RE: Name/label/id metadata: how do I make it go away Thread-Topic: Name/label/id metadata: how do I make it go away Thread-Index: AQHQ9JzHyu4NGbe190ufZB2ZpGj8Vp5H1yIAgAApr4CAAC1l4A== Date: Tue, 22 Sep 2015 08:15:05 +0000 Message-ID: References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> In-Reply-To: <5600F0DF.8000805@stankevitz.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 08:17:37 -0000 >Warren/Freddie, >Thank you for your replies. >On 9/21/15 8:41 PM, Warren Block wrote: >>> - gpt id >> >> A system-assigned ID in the GPT metadata. Requires GPT partitioning. >>> I have a zfs pool of "entire disks". "zpool status" shows some disks=20 >>> with their daX name (which I prefer) and some with a hideously ugly=20 >>> name such as DISK-%20%20%20%20%20-WD-WMC4NOH1ASDF >> >> Disable those labels with kern.geom.label.gptid.enable=3D"0" in=20 >> /boot/loader.conf. >Combining your two statements quoted above, I believe I can conclude that = my ZFS "whole disk" drives must have some remnants of GPT left >over from t= heir previous lives (namely the system-assigned ID in the GPT metadata). >Surprisingly, these apparently GPT-supplied labels appear to "go away"=20 >when I issue a "glabel stop". I would not expect this given that Warren e= xplained that glabels (whose metadata are stored at the end of the >device = and completely outside the virtual device) are not the same as GPT labels (= whose metadata are stored within the device on the GPT >metadata). >I believe one of the following must be true: >1. It is possible to use "glabel stop" to disable a "GPT system-assigned I= D" -- even though glabel is a tool for manipulating another style of labels= . >2. "glabel stop" only affects glabels. In my case my drives must contain = "glables" and not "GPT system-assigned IDs" as Warren guessed. >3. I misunderstood and glabels/GPS system-assigned IDs are really the same= thing. >Thank you again, >Chris The diskid entries have nothing to do with GPT. I suspect your disks do not= have any partitioning. These labels are generated automatically based on t= he actual vendor supplied ID for each disk. The glabel command allows you to assign a label to a GEOM device, which wil= l then appear as /dev/label/name. I personally dislike these as they are no= t portable and are one block smaller than the original device, which is sti= ll accessible. It appears glabel is also responsible for identifying various other "labell= ing" methods that weren't necessary created using the utility. For instance= it will create /dev/ufsid/X devices, where X is the ID of a UFS filesystem= it has found on a GEOM device. It also seems to be responsible for creatin= g /dev/gpt entries as well as /dev/diskid, which is possibly why they disap= pear when you run glabel stop. Most of this is mentioned in the man page, a= lthough it doesn't mention the diskid entries; There does however appear to= be a geom/label/g_label_disk_ident.c source file which implements the /dev= /diskid/X functionality. The thing to be clear on is that although glabel is responsible for creatin= g the device nodes for most of these labels, the way they are assigned is v= ery different=20 diskid - Automatic based on the ID of the disk - so should reference an ent= ire disk gpt - Assigned manually to a partition by GPT partitioning a disk and creat= ing labelled partitions gptid - Automatic when you partition a disk with GPT and create partitions = (every GPT partition has a unique GUID) glabel - Assigned to any disk or partition manually by using the glabel com= mand Interestingly, looking though sysctl, I see options to turn off most label = devices, but nothing mentions diskid. I don't know if this is actually mis= sing, or if one of the existing sysctls also turns them off. (Surely there = should ideally be a kern.geom.label.diskid.enable?) kern.geom.label.ext2fs.enable: 1 kern.geom.label.iso9660.enable: 1 kern.geom.label.msdosfs.enable: 1 kern.geom.label.ntfs.enable: 1 kern.geom.label.reiserfs.enable: 1 kern.geom.label.ufs.enable: 1 kern.geom.label.ufsid.enable: 1 kern.geom.label.gptid.enable: 1 kern.geom.label.gpt.enable: 1 Regards, Matt Churchyard From owner-freebsd-fs@freebsd.org Tue Sep 22 09:44:41 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1FD58A06866 for ; Tue, 22 Sep 2015 09:44:41 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-outbound.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C5FCC18F9 for ; Tue, 22 Sep 2015 09:44:40 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) by smtp-outbound.userve.net (8.15.1/8.15.1) with ESMTPS id t8M9iXsY044103 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 22 Sep 2015 10:44:34 +0100 (BST) (envelope-from matt.churchyard@userve.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1442915075; bh=2cexJrQ6DjpzenrUtP/8dLv3A3jOPEQTegp0OoRYETQ=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=IpmWnaAMKVtljoIXcilRGxs/qYEqbsu/w8o+A3lNSzJq2jcw45qnVBgbTZSxvLZJo 7uwypef/l0K+jjj5csHtFj0+fQtATNig/ig0Q5ByowJH0Q/TO0p3L6rvvxLPXpZueb KoLpyEFT+EB6I5OFa3GBvhzcr54s2dyJgh+HhQTw= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 22 Sep 2015 10:44:28 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Tue, 22 Sep 2015 10:44:27 +0100 From: Matt Churchyard To: FreeBSD FS Subject: RE: Name/label/id metadata: how do I make it go away Thread-Topic: Name/label/id metadata: how do I make it go away Thread-Index: AQHQ9JzHyu4NGbe190ufZB2ZpGj8Vp5H1yIAgAApr4CAAC1l4IAAHoTA Date: Tue, 22 Sep 2015 09:44:27 +0000 Message-ID: <5cd22ba8f67a46a0a1f77b4646f3eaa5@SERVER.ad.usd-group.com> References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 09:44:41 -0000 >Warren/Freddie, >Thank you for your replies. >On 9/21/15 8:41 PM, Warren Block wrote: >>> - gpt id >> >> A system-assigned ID in the GPT metadata. Requires GPT partitioning. >>> I have a zfs pool of "entire disks". "zpool status" shows some=20 >>> disks with their daX name (which I prefer) and some with a hideously=20 >>> ugly name such as DISK-%20%20%20%20%20-WD-WMC4NOH1ASDF >> >> Disable those labels with kern.geom.label.gptid.enable=3D"0" in=20 >> /boot/loader.conf. >Combining your two statements quoted above, I believe I can conclude that = my ZFS "whole disk" drives must have some remnants of GPT left >over from t= heir previous lives (namely the system-assigned ID in the GPT metadata). >Surprisingly, these apparently GPT-supplied labels appear to "go away"=20 >when I issue a "glabel stop". I would not expect this given that Warren e= xplained that glabels (whose metadata are stored at the end of the >device = and completely outside the virtual device) are not the same as GPT labels (= whose metadata are stored within the device on the GPT >metadata). >I believe one of the following must be true: >1. It is possible to use "glabel stop" to disable a "GPT system-assigned I= D" -- even though glabel is a tool for manipulating another style of labels= . >2. "glabel stop" only affects glabels. In my case my drives must contain = "glables" and not "GPT system-assigned IDs" as Warren guessed. >3. I misunderstood and glabels/GPS system-assigned IDs are really the same= thing. >Thank you again, >Chris >The diskid entries have nothing to do with GPT. I suspect your disks do no= t have any partitioning. These labels are generated automatically >based on= the actual vendor supplied ID for each disk. >The glabel command allows you to assign a label to a GEOM device, which wi= ll then appear as /dev/label/name. I personally dislike these as >they are = not portable and are one block smaller than the original device, which is s= till accessible. >It appears glabel is also responsible for identifying various other "label= ling" methods that weren't necessary created using the utility. For >instan= ce it will create /dev/ufsid/X devices, where X is the ID of a UFS filesyst= em it has found on a GEOM device. It also seems to be >responsible for crea= ting /dev/gpt entries as well as /dev/diskid, which is possibly why they di= sappear when you run glabel stop. Most of this is >mentioned in the man pag= e, although it doesn't mention the diskid entries; There does however appea= r to be a >geom/label/g_label_disk_ident.c source file >which implements th= e /dev/diskid/X functionality. >The thing to be clear on is that although glabel is responsible for creati= ng the device nodes for most of these labels, the way they are assigned is = >very different=20 >diskid - Automatic based on the ID of the disk - so should reference an en= tire disk gpt - Assigned manually to a partition by GPT partitioning a >dis= k and creating labelled partitions gptid - Automatic when you partition a d= isk with GPT and create partitions (every GPT partition has a unique >GUID)= glabel - Assigned to any disk or partition manually by using the glabel co= mmand >Interestingly, looking though sysctl, I see options to turn off most label= devices, but nothing mentions diskid. I don't know if this is actually >m= issing, or if one of the existing sysctls also turns them off. (Surely ther= e should ideally be a kern.geom.label.diskid.enable?) >kern.geom.label.ext2fs.enable: 1 >kern.geom.label.iso9660.enable: 1 >kern.geom.label.msdosfs.enable: 1 >kern.geom.label.ntfs.enable: 1 >kern.geom.label.reiserfs.enable: 1 >kern.geom.label.ufs.enable: 1 >kern.geom.label.ufsid.enable: 1 >kern.geom.label.gptid.enable: 1 >kern.geom.label.gpt.enable: 1 Ignore that last bit. Just found the sysctl to turn off the diskid entries = - kern.geom.label.disk_ident.enable: 1 Regards, Matt Churchyard From owner-freebsd-fs@freebsd.org Tue Sep 22 10:32:15 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE25BA06331 for ; Tue, 22 Sep 2015 10:32:15 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 57C95158A; Tue, 22 Sep 2015 10:32:15 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: by wicgb1 with SMTP id gb1so153141297wic.1; Tue, 22 Sep 2015 03:32:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8U8QiWxGDqawZjkqjJ/9wHct9v2idHCxcHcHxaM5A2Q=; b=yl1WDn5H7T4mtkNpz6NeApwhgFHM48N0c0C3K2rBslNt7iaNwmo2fDI2sRSwjffqLd Oj3tm1qHZ91iwrnuzYyJBEt/+omu7K2oc9kVmnI7s87VgOWMAx0WF9HMaB3wCb3H6TSh Rf9jkpd/CFXlhb43XASzsvvthbiZoH230mRyvT/YetquJ6GxSDl4BQXmVdQgs8QwtTcq DotmJSeg0YNX+LfqZ+G7DtDT4tOGqUTFXBOeXqDEvnJ5qw/TK2KuH+nbzj0gVzGYXgJm 9IhGpuUsXywRgFEXvrRLIKxYuOJF2Z2BbKDqfDhjl9cnrgEvBuMpEcMxueKfHYy0E4kG o5OA== MIME-Version: 1.0 X-Received: by 10.180.198.109 with SMTP id jb13mr19343550wic.17.1442917933760; Tue, 22 Sep 2015 03:32:13 -0700 (PDT) Received: by 10.28.125.18 with HTTP; Tue, 22 Sep 2015 03:32:13 -0700 (PDT) In-Reply-To: <5600798B.3010208@FreeBSD.org> References: <20150921170216.GA98888@blazingdot.com> <20150921211335.GB41102@server.rulingia.com> <5600798B.3010208@FreeBSD.org> Date: Tue, 22 Sep 2015 11:32:13 +0100 Message-ID: Subject: Re: ZFS cpu requirements, with/out compression and/or dedup From: krad To: Matthew Seaman Cc: FreeBSD FS Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 10:32:15 -0000 or far more easily do an rsync from the old to the new with the remove-source-files option, and then drop the old dataset at the end On 21 September 2015 at 22:41, Matthew Seaman wrote: > On 21/09/2015 22:13, Peter Jeremy wrote: > > In general, the downsides of dedup outweigh the benefits. If you already > > have the data in ZFS, you can use 'zdb -S' to see what effect rebuilding > > the pool with dedup enabled would have - how much disk space you will > save > > and how big the DDT is (and hence how much RAM you will need). If you > can > > afford it, make sure you keep good backups, enable DDT and be ready to > nuke > > the pool and restore from backups if dedup doesn't work out. > > Nuking the entire pool is a little heavy handed. Dedup can be turned on > and off on a per-ZFS basis. If you've a ZFS that had dedup enabled, you > can remove the effects by zfs send / zfs recv to create a pristine > un-deduped copy of the data, destroy the original zfs and rename the new > one to take its place. Of course, this depends on your having enough > free space in the pool to be able to duplicate (and then some) that ZFS. > > Failing that, you might be able to 'zpool split' if your pool is > composed entirely of mirrors. So long as you're able to do without > resilience for a while this basically doubles the space you have > available to play with. You can then destroy the contents of one of the > split zpools, and zfs send the data over from the other split pool. > Unfortunately there isn't a reciprocal 'zfs rejoin' command that undoes > the splitting, so you'll have to destroy one of the splits and re-add > the constituent devices back to restore the mirroring in the other > split. Which is a delicate operations and not one which is forgiving of > mistakes. > > And failing that, you can start pushing data over the network, but > that's hardly different to restoring from backup. However, either of > the first two choices should be significantly faster if you have large > quantities of data to handle. > > Cheers, > > Matthew > > > From owner-freebsd-fs@freebsd.org Tue Sep 22 14:40:39 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78AF3A075FB for ; Tue, 22 Sep 2015 14:40:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 65BBE1280 for ; Tue, 22 Sep 2015 14:40:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8MEedeC031346 for ; Tue, 22 Sep 2015 14:40:39 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 198457] zfs acl lost after zfs send-receive. Kernel panic Date: Tue, 22 Sep 2015 14:40:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: o.bende@gmail.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 14:40:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198457 o.bende@gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |o.bende@gmail.com --- Comment #1 from o.bende@gmail.com --- I experienced the same problem in 2014 and tried today again with the same result. Are there any news regarding this? I cannot believe, there are only 3 persons in the world having this issue. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Tue Sep 22 14:42:08 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13B6EA077D2 for ; Tue, 22 Sep 2015 14:42:08 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C72CC152B for ; Tue, 22 Sep 2015 14:42:07 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.15.2/8.15.2) with ESMTPS id t8MEg56B044977 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 22 Sep 2015 08:42:05 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.15.2/8.15.2/Submit) with ESMTP id t8MEg5RK044974; Tue, 22 Sep 2015 08:42:05 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Tue, 22 Sep 2015 08:42:05 -0600 (MDT) From: Warren Block To: Chris Stankevitz cc: FreeBSD Filesystems Subject: Re: Name/label/id metadata: how do I make it go away In-Reply-To: <5600F0DF.8000805@stankevitz.com> Message-ID: References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Tue, 22 Sep 2015 08:42:05 -0600 (MDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 14:42:08 -0000 On Mon, 21 Sep 2015, Chris Stankevitz wrote: > Warren/Freddie, > > Thank you for your replies. > > On 9/21/15 8:41 PM, Warren Block wrote: >>> - gpt id >> >> A system-assigned ID in the GPT metadata. Requires GPT partitioning. ZFS on bare devices uses an EFI label that is similar or related to GPT. Those might be detected and displayed even without GPT, but I'm not sure. >>> I have a zfs pool of "entire disks". "zpool status" shows some disks >>> with their daX name (which I prefer) and some with a hideously ugly >>> name such as DISK-%20%20%20%20%20-WD-WMC4NOH1ASDF >> >> Disable those labels with kern.geom.label.gptid.enable="0" in >> /boot/loader.conf. > > Combining your two statements quoted above, I believe I can conclude that my > ZFS "whole disk" drives must have some remnants of GPT left over from their > previous lives (namely the system-assigned ID in the GPT metadata). > > Surprisingly, these apparently GPT-supplied labels appear to "go away" when I > issue a "glabel stop". I would not expect this given that Warren explained > that glabels (whose metadata are stored at the end of the device and > completely outside the virtual device) are not the same as GPT labels (whose > metadata are stored within the device on the GPT metadata). "glabel" is both a command that creates manual labels and a GEOM class that creates the entries in /dev when device labels of all types are found. Stopping the label class stops the display of all labels. glabel(8) is worth the read. From owner-freebsd-fs@freebsd.org Tue Sep 22 14:44:36 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D9C1A07892 for ; Tue, 22 Sep 2015 14:44:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5A6541621 for ; Tue, 22 Sep 2015 14:44:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8MEiaW7040998 for ; Tue, 22 Sep 2015 14:44:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 198457] zfs acl lost after zfs send-receive. Kernel panic Date: Tue, 22 Sep 2015 14:44:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: o.bende@gmail.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 14:44:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198457 --- Comment #2 from o.bende@gmail.com --- Link to other Bug-Tickets: https://bugs.freenas.org/issues/5225 https://forums.freenas.org/index.php?threads/zfs-replication-corrupts-entire-zfs-volume-warning.17566/#post-126243 So far, only posted in Freenas bug-trackers - but same issue here. Probably, this is a similar/same issue: http://markmail.org/message/4nl4dzkmuo7gidlu -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Tue Sep 22 14:50:52 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CFECFA07AD6 for ; Tue, 22 Sep 2015 14:50:52 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-vk0-x22c.google.com (mail-vk0-x22c.google.com [IPv6:2607:f8b0:400c:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 88354194A for ; Tue, 22 Sep 2015 14:50:52 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by vkgd64 with SMTP id d64so7962071vkg.0 for ; Tue, 22 Sep 2015 07:50:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DJIJF8D/RYTzyRSJn9QxdI4W8Fe+vDTAlpvD8YzgWPI=; b=ZuIPDGTSQmPyuKjBQKU9oYTT2UdokfvWR5TTrflVStc1yCNHIQGWDh5olEcOS0RTqG NgvfuIXk683uRW26/d6T274GrhBnXROPKaGP7vFYfUmZjXlanrCLzXXYIKXpUoNmGArm 2RXit27nwc6CKpTUY3OhBg0frFOfyV2nHUGynCWPQRbSwG7IjoVVyjIYcvtQLBCnfCyZ qjn9+P8TfR4mnEmrAqSBXi/PZvztTMkCRJUkArbgyY2DjcLybvo3FTnzcZlRGOUqJ2Rr MJy9EbknknRFkt3vuLVpz7Ztv65eyIf9hZ+I4gguZhiQBmnnp80qXcM4s402dXklIPnw LGLg== MIME-Version: 1.0 X-Received: by 10.31.9.202 with SMTP id 193mr12357887vkj.155.1442933451566; Tue, 22 Sep 2015 07:50:51 -0700 (PDT) Received: by 10.31.200.7 with HTTP; Tue, 22 Sep 2015 07:50:51 -0700 (PDT) In-Reply-To: <5cd22ba8f67a46a0a1f77b4646f3eaa5@SERVER.ad.usd-group.com> References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> <5cd22ba8f67a46a0a1f77b4646f3eaa5@SERVER.ad.usd-group.com> Date: Tue, 22 Sep 2015 07:50:51 -0700 Message-ID: Subject: Re: Name/label/id metadata: how do I make it go away From: Freddie Cash To: Matt Churchyard Cc: FreeBSD FS Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 14:50:53 -0000 On Tue, Sep 22, 2015 at 2:44 AM, Matt Churchyard via freebsd-fs < freebsd-fs@freebsd.org> wrote: > >Warren/Freddie, > > >Thank you for your replies. > > >On 9/21/15 8:41 PM, Warren Block wrote: > >>> - gpt id > >> > >> A system-assigned ID in the GPT metadata. Requires GPT partitioning. > > >>> I have a zfs pool of "entire disks". "zpool status" shows some > >>> disks with their daX name (which I prefer) and some with a hideously > >>> ugly name such as DISK-%20%20%20%20%20-WD-WMC4NOH1ASDF > >> > >> Disable those labels with kern.geom.label.gptid.enable=3D"0" in > >> /boot/loader.conf. > > >Combining your two statements quoted above, I believe I can conclude tha= t > my ZFS "whole disk" drives must have some remnants of GPT left >over from > their previous lives (namely the system-assigned ID in the GPT metadata). > > >Surprisingly, these apparently GPT-supplied labels appear to "go away" > >when I issue a "glabel stop". I would not expect this given that Warren > explained that glabels (whose metadata are stored at the end of the >devi= ce > and completely outside the virtual device) are not the same as GPT labels > (whose metadata are stored within the device on the GPT >metadata). > > >I believe one of the following must be true: > > >1. It is possible to use "glabel stop" to disable a "GPT system-assigned > ID" -- even though glabel is a tool for manipulating another style of > labels. > > >2. "glabel stop" only affects glabels. In my case my drives must contai= n > "glables" and not "GPT system-assigned IDs" as Warren guessed. > > >3. I misunderstood and glabels/GPS system-assigned IDs are really the > same thing. > > >Thank you again, > > >Chris > > >The diskid entries have nothing to do with GPT. I suspect your disks do > not have any partitioning. These labels are generated automatically >base= d > on the actual vendor supplied ID for each disk. > > >The glabel command allows you to assign a label to a GEOM device, which > will then appear as /dev/label/name. I personally dislike these as >they > are not portable and are one block smaller than the original device, whic= h > is still accessible. > > >It appears glabel is also responsible for identifying various other > "labelling" methods that weren't necessary created using the utility. For > >instance it will create /dev/ufsid/X devices, where X is the ID of a UFS > filesystem it has found on a GEOM device. It also seems to be >responsibl= e > for creating /dev/gpt entries as well as /dev/diskid, which is possibly w= hy > they disappear when you run glabel stop. Most of this is >mentioned in th= e > man page, although it doesn't mention the diskid entries; There does > however appear to be a >geom/label/g_label_disk_ident.c source file >whic= h > implements the /dev/diskid/X functionality. > > >The thing to be clear on is that although glabel is responsible for > creating the device nodes for most of these labels, the way they are > assigned is >very different > > >diskid - Automatic based on the ID of the disk - so should reference an > entire disk gpt - Assigned manually to a partition by GPT partitioning a > >disk and creating labelled partitions gptid - Automatic when you partiti= on > a disk with GPT and create partitions (every GPT partition has a unique > >GUID) glabel - Assigned to any disk or partition manually by using the > glabel command > > >Interestingly, looking though sysctl, I see options to turn off most > label devices, but nothing mentions diskid. I don't know if this is > actually >missing, or if one of the existing sysctls also turns them off= . > (Surely there should ideally be a kern.geom.label.diskid.enable?) > > >kern.geom.label.ext2fs.enable: 1 > >kern.geom.label.iso9660.enable: 1 > >kern.geom.label.msdosfs.enable: 1 > >kern.geom.label.ntfs.enable: 1 > >kern.geom.label.reiserfs.enable: 1 > >kern.geom.label.ufs.enable: 1 > >kern.geom.label.ufsid.enable: 1 > >kern.geom.label.gptid.enable: 1 > >kern.geom.label.gpt.enable: 1 > > Ignore that last bit. Just found the sysctl to turn off the diskid entrie= s > - > > kern.geom.label.disk_ident.enable: 1 > =E2=80=8BThat must be new in 10.x. In 9.x (and presumably 8.x), the gptid sysctl disables the diskid labels. Guessing in 10.x they split the two into separate sysctls? --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@freebsd.org Tue Sep 22 19:12:50 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCBBFA0763B for ; Tue, 22 Sep 2015 19:12:50 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from mango.stankevitz.com (mango.stankevitz.com [208.79.93.194]) by mx1.freebsd.org (Postfix) with ESMTP id A9F8F11A5 for ; Tue, 22 Sep 2015 19:12:50 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from Chriss-MacBook-Pro.local (209-203-101-124.static.twtelecom.net [209.203.101.124]) (using TLSv1.2 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mango.stankevitz.com (Postfix) with ESMTPSA id 67E7311B7; Tue, 22 Sep 2015 12:12:44 -0700 (PDT) Subject: Re: Name/label/id metadata: how do I make it go away To: Matt Churchyard References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> Cc: FreeBSD FS From: Chris Stankevitz Message-ID: <5601A82A.7040304@stankevitz.com> Date: Tue, 22 Sep 2015 12:12:42 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 19:12:50 -0000 On 9/22/15 1:15 AM, Matt Churchyard wrote: > The thing to be clear on is that although glabel is responsible for > creating the device nodes for most of these labels, the way they are > assigned is very different Matt, Thank you for your explanation. I will read the glabel(8) man page seeing that it is almost certainly responsible for what is going on -- even though glabel was not used to create the diskid. > diskid - Automatic based on the ID of the disk - so should reference > an entire disk I want to read the man page or source for whatever assigns diskid for glabel to subsequently advertise. That process, for some reason, only assigns diskids for 10 of my 22 drives. All drives are the same model number, although the drives have different histories. (some drives lived in FreeNAS in a former life) My running theory is that there is indeed some kind of metadata stored on the disk indicating a preference for or against diskid... but that it is "obfuscated". For example "if a disk formally had GPT, but the primary GPT table was overwritten and the secondary still exists at the end of the disk then diskid is/isn't used." Otherwise I cannot fathom why some of my identical disks advertise diskid while others do not. Chris From owner-freebsd-fs@freebsd.org Tue Sep 22 19:31:02 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 401E1A06168 for ; Tue, 22 Sep 2015 19:31:02 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from mango.stankevitz.com (mango.stankevitz.com [208.79.93.194]) by mx1.freebsd.org (Postfix) with ESMTP id 2D5081112 for ; Tue, 22 Sep 2015 19:31:01 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from Chriss-MacBook-Pro.local (209-203-101-124.static.twtelecom.net [209.203.101.124]) (using TLSv1.2 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mango.stankevitz.com (Postfix) with ESMTPSA id 2C41511C0; Tue, 22 Sep 2015 12:31:01 -0700 (PDT) Subject: Re: Name/label/id metadata: how do I make it go away To: Warren Block References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> Cc: FreeBSD Filesystems From: Chris Stankevitz Message-ID: <5601AC74.2090209@stankevitz.com> Date: Tue, 22 Sep 2015 12:31:00 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 19:31:02 -0000 On 9/22/15 7:42 AM, Warren Block wrote: > ZFS on bare devices uses an EFI label that is similar or related to GPT. > Those might be detected and displayed even without GPT, but I'm not sure. Thank you. I'm using google to figure out what FreeBSD utility will allow me to query the EFI label. After I find that I'll read the man page. Then I'll use the tool to see if I can read/write the label. > "glabel" is both a command that creates manual labels and a GEOM class > that creates the entries in /dev when device labels of all types are > found. Stopping the label class stops the display of all labels. > glabel(8) is worth the read. Got it, thank you. Reading glabel(8) now. Chris From owner-freebsd-fs@freebsd.org Tue Sep 22 19:57:36 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B09F0A06F4E for ; Tue, 22 Sep 2015 19:57:36 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from mango.stankevitz.com (mango.stankevitz.com [208.79.93.194]) by mx1.freebsd.org (Postfix) with ESMTP id 9DBC91139 for ; Tue, 22 Sep 2015 19:57:36 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from Chriss-MacBook-Pro.local (209-203-101-124.static.twtelecom.net [209.203.101.124]) (using TLSv1.2 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mango.stankevitz.com (Postfix) with ESMTPSA id DBB6311C7; Tue, 22 Sep 2015 12:57:35 -0700 (PDT) Subject: Re: Name/label/id metadata: how do I make it go away To: Matt Churchyard References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> <5601A82A.7040304@stankevitz.com> Cc: FreeBSD FS From: Chris Stankevitz Message-ID: <5601B2AF.7040306@stankevitz.com> Date: Tue, 22 Sep 2015 12:57:35 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <5601A82A.7040304@stankevitz.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 19:57:36 -0000 On 9/22/15 12:12 PM, Chris Stankevitz wrote: > I want to read the man page or source for whatever assigns diskid for > glabel to subsequently advertise. That process, for some reason, only > assigns diskids for 10 of my 22 drives. All drives are the same model > number, although the drives have different histories. (some drives > lived in FreeNAS in a former life) According to 'geom disk list', each of my 22 drives has an "ident", but only some of them have "labels". "ident" is turned into a "label" via a process called "tasting": sys/geom/label/g_label_disk_ident.c sys/geom/label/g_label.c Although it is not yet clear to me why apparently only some of my disks are "tasted". Chris From owner-freebsd-fs@freebsd.org Tue Sep 22 20:19:12 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5F829CEDF2 for ; Tue, 22 Sep 2015 20:19:12 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-wi0-x230.google.com (mail-wi0-x230.google.com [IPv6:2a00:1450:400c:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5522F1918 for ; Tue, 22 Sep 2015 20:19:12 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by wicfx3 with SMTP id fx3so40951706wic.0 for ; Tue, 22 Sep 2015 13:19:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=QRjnnjnV3LGHUzo0ISLMFVYtGcUwT8mDTaa7cVUoS8s=; b=o9gsMYjflHwXrRZrr8CwUyB7GPuYkqrjxVbYGgHSCCUgyCzpFXtpJn7YOsQMwotQsL SpmvNzupWTCSg2JaHX3c1FFE9vUiuAm4joE1fPm+6CtR2cjq7kBQNUDVwMH2aHqHQ+Jg KJW2pwJ8RGylFdRT8N5TxS8O+g9dthXBJcwA0CB7Ag0ANvHEro20zixKwtEZQTxSBEUe bz9kVbcD8tLEwgXXxKUEEcT9qcuUa1JwGzXpjB4ExQukzF7uKPTc+z0qeoh9UPEbQnN6 XA90zqy/9d5DQRRd14NUpg/4C/KkNrPaJRXzJx8fIuGrGvSHvpQN7ejnCmOiy9BvQF0U pwZA== X-Received: by 10.194.52.106 with SMTP id s10mr29892958wjo.35.1442953150877; Tue, 22 Sep 2015 13:19:10 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by smtp.gmail.com with ESMTPSA id fx2sm4868592wib.24.2015.09.22.13.19.09 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Tue, 22 Sep 2015 13:19:09 -0700 (PDT) Date: Tue, 22 Sep 2015 22:19:07 +0200 From: Mateusz Guzik To: Kevin Day Cc: freebsd-fs@freebsd.org Subject: Re: Neutered devices in jails (per FS flag?) Message-ID: <20150922201907.GA27724@dft-labs.eu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 20:19:12 -0000 On Wed, Sep 16, 2015 at 11:30:33AM -0500, Kevin Day wrote: > We’re currently using jails to allow servers to copy backups of themselves to a central backup server. The problem we’re having is with mknod/devices. Currently jails don’t allow device files to be created, which makes sense - you don’t want them to be able to bypass the jail by opening /dev/kmem or something. We want jails to be able to create device files, just not be able to open/use them. > > Has anyone given any thought to changing this behavior? Allowing jails to create/manipulate device files, but not actually opening them? I.e. instead of returning EPERM on creating the device, instead return EPERM on opening it? This would likely need to be a filesystem flag, because jails still require some devices to work (a separate devfs mount or something). We could make the jail’s /dev read only or use devfs so those devices still work, but have the parent jail directory with a “noopendev” flag or something similar. > > Has anyone gone down this path before? > Let's lay down some facts to make things clear. 1. device nodes on regular filesystems are not treated as devices by the kernel 2. device visibility in devfs is controlled with appropriate rules. mknoding a device will make it appear, regardless of presence of a 'hide' rule So, mknod in question /on devfs/ would be useful to make stuff reappear if it was deleted by accident, i.e. its a nice little feature. Allowing jailed root to make explicitly hidden devices visible is a complete non-starter (regardless of whether it is allowed to use them) so this would have to be plugged. Allowing jailed root to mknod on regular filesystems by defualt is also a non-starter because said filesystems may be exported with nfs and the other party possibly forgot about nodev and actually respects device nodes. Further, you can mknod more than just a device so that would have to be audited. So, to summarize, this can be done. So what is needed for such a feature to hit the tree: 1. it would have to be an opt-in thingy (similar to how e.g. sysvipc is handled) - a trivial change 2. arbitrary device creation on devfs would have to be disabled if the user is jailed - likely a trivial change 3. someone has to audit mknod - unclear I'm not up to the task at the moment though. I'm happy to take a look at patches for 1 and 2, but I'm not committing anything without point 3 being executed (and I'm not touching it for now). -- Mateusz Guzik From owner-freebsd-fs@freebsd.org Tue Sep 22 20:24:29 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9F7FA021C0 for ; Tue, 22 Sep 2015 20:24:29 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 80FBE1E50 for ; Tue, 22 Sep 2015 20:24:29 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 1A3632052F for ; Tue, 22 Sep 2015 16:24:28 -0400 (EDT) Received: from web3 ([10.202.2.213]) by compute6.internal (MEProxy); Tue, 22 Sep 2015 16:24:28 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=5OqsegE8vK91HBJ SSgSSZi9A7vM=; b=TBEGAFXVvDSwapYr0QUudOXWzL4qIPlwEK4xC9+Ny3Vy/Kx 8PVif56bLGkhCLuV21+PKaHNBb0BERrMCiSwyvu814zEL4hNBFjCr0/q80dMlAek ofMlsHF0ptjtjaFHcXak0Nd7JYtoWaMkLjNGqJdYWbEeHLrG4iL2xSB0jE5A= Received: by web3.nyi.internal (Postfix, from userid 99) id E4638102AAC; Tue, 22 Sep 2015 16:24:27 -0400 (EDT) Message-Id: <1442953467.1530841.390840385.067EC1E4@webmail.messagingengine.com> X-Sasl-Enc: hBX4z90ZtLzK2dSeb4lnX1pOrVZGRszeGQA8y0GFQOI8 1442953467 From: Mark Felder To: "InterNetX - Juergen Gotteswinter" , freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-39fe9c8e Subject: Re: [HAST] ZFS, many disks, write order Date: Tue, 22 Sep 2015 15:24:27 -0500 In-Reply-To: <560102BD.6040703@internetx.com> References: <560102BD.6040703@internetx.com> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 20:24:29 -0000 On Tue, Sep 22, 2015, at 02:26, InterNetX - Juergen Gotteswinter wrote: > One of the great Hast Features is that it Hides Disk Failures from ZFS, > this has been already discussed on this List a few Months ago. > > Single ZIL Flash Drive is also not such a great Idea. I whould suggest > to start from scratch with "ZFS Best Practices". > A single ZIL is not a single point of failure for a few years now. Most people do not have a ton of data going through the ZIL and if it dies you only lose those transactions in flight, not your entire pool. -- Mark Felder ports-secteam member feld@FreeBSD.org From owner-freebsd-fs@freebsd.org Tue Sep 22 20:29:28 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF2BDA023C1 for ; Tue, 22 Sep 2015 20:29:28 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 894EF106F for ; Tue, 22 Sep 2015 20:29:28 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.15.2/8.15.2) with ESMTPS id t8MKTQpt034228 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 22 Sep 2015 14:29:26 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.15.2/8.15.2/Submit) with ESMTP id t8MKTQoQ034225; Tue, 22 Sep 2015 14:29:26 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Tue, 22 Sep 2015 14:29:26 -0600 (MDT) From: Warren Block To: Chris Stankevitz cc: FreeBSD Filesystems Subject: Re: Name/label/id metadata: how do I make it go away In-Reply-To: <5601AC74.2090209@stankevitz.com> Message-ID: References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> <5601AC74.2090209@stankevitz.com> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Tue, 22 Sep 2015 14:29:26 -0600 (MDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 20:29:28 -0000 On Tue, 22 Sep 2015, Chris Stankevitz wrote: > On 9/22/15 7:42 AM, Warren Block wrote: >> ZFS on bare devices uses an EFI label that is similar or related to GPT. >> Those might be detected and displayed even without GPT, but I'm not sure. > > Thank you. I'm using google to figure out what FreeBSD utility will allow me > to query the EFI label. After I find that I'll read the man page. Then I'll > use the tool to see if I can read/write the label. Do not mess with ZFS's stuff. It might be touchy about that. From owner-freebsd-fs@freebsd.org Tue Sep 22 21:01:24 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6841DA03769 for ; Tue, 22 Sep 2015 21:01:24 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2ABE91B9E for ; Tue, 22 Sep 2015 21:01:24 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.15.2/8.15.2) with ESMTPS id t8ML1MM4042172 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 22 Sep 2015 15:01:23 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.15.2/8.15.2/Submit) with ESMTP id t8ML1Mad042169; Tue, 22 Sep 2015 15:01:22 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Tue, 22 Sep 2015 15:01:22 -0600 (MDT) From: Warren Block To: Chris Stankevitz cc: Matt Churchyard , FreeBSD FS Subject: Re: Name/label/id metadata: how do I make it go away In-Reply-To: <5601B2AF.7040306@stankevitz.com> Message-ID: References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> <5601A82A.7040304@stankevitz.com> <5601B2AF.7040306@stankevitz.com> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Tue, 22 Sep 2015 15:01:23 -0600 (MDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 21:01:24 -0000 On Tue, 22 Sep 2015, Chris Stankevitz wrote: > On 9/22/15 12:12 PM, Chris Stankevitz wrote: >> I want to read the man page or source for whatever assigns diskid for >> glabel to subsequently advertise. That process, for some reason, only >> assigns diskids for 10 of my 22 drives. All drives are the same model >> number, although the drives have different histories. (some drives >> lived in FreeNAS in a former life) > > > According to 'geom disk list', each of my 22 drives has an "ident", but only > some of them have "labels". > > "ident" is turned into a "label" via a process called "tasting": > sys/geom/label/g_label_disk_ident.c > sys/geom/label/g_label.c > > Although it is not yet clear to me why apparently only some of my disks are > "tasted". Labels on devices that are mounted are "withered" and disappear. From owner-freebsd-fs@freebsd.org Tue Sep 22 21:43:35 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A99ABA06D35 for ; Tue, 22 Sep 2015 21:43:35 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from mango.stankevitz.com (mango.stankevitz.com [208.79.93.194]) by mx1.freebsd.org (Postfix) with ESMTP id 94F6E1151 for ; Tue, 22 Sep 2015 21:43:35 +0000 (UTC) (envelope-from chris@stankevitz.com) Received: from Chriss-MacBook-Pro.local (209-203-101-124.static.twtelecom.net [209.203.101.124]) (using TLSv1.2 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mango.stankevitz.com (Postfix) with ESMTPSA id 33A9311E5; Tue, 22 Sep 2015 14:43:34 -0700 (PDT) Subject: Re: Name/label/id metadata: how do I make it go away To: Warren Block References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> <5601A82A.7040304@stankevitz.com> <5601B2AF.7040306@stankevitz.com> Cc: Matt Churchyard , FreeBSD FS From: Chris Stankevitz Message-ID: <5601CB85.8070400@stankevitz.com> Date: Tue, 22 Sep 2015 14:43:33 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 21:43:35 -0000 On 9/22/15 2:01 PM, Warren Block wrote: >> According to 'geom disk list', each of my 22 drives has an "ident", >> but only some of them have "labels". >> >> "ident" is turned into a "label" via a process called "tasting": >> sys/geom/label/g_label_disk_ident.c >> sys/geom/label/g_label.c >> >> Although it is not yet clear to me why apparently only some of my >> disks are "tasted". > > Labels on devices that are mounted are "withered" and disappear. Thank you. Indeed when I export the pool, all of the drives become listed in /dev/diskid. When I re-import the pool, some of the drives "wither" and are removed from /dev/diskid. These withered drives appear as daX in "zpool status". The drives that do not wither remain in /dev/diskid and appear in "zpool status" with their /dev/diskid/X names. Q: Why do my identical zpool drives behave differently? Please let me know if my hypothesis is plausible: ZFS sees both the /dev/daX "consumers" and the /dev/diskid/X "providers". ZFS is randomly selecting which to use to import the pool. Sometimes it picks "consumers" and sometimes "providers". When ZFS picks a consumer, the provider is withered away. When ZFS picks a provider, the provider remains. And if I want to dig deeper into root cause I can ask ZFS "why do you sometimes select from the consumer collection and sometimes from the provider collection when putting a pool together". Or if I don't want to dig deeper I can "deal with it" or I can disable diskid using kern.geom.label.disk_ident.enable Thank you again, Chris From owner-freebsd-fs@freebsd.org Tue Sep 22 22:04:52 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70A04A076FA for ; Tue, 22 Sep 2015 22:04:52 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 34D4E1B39; Tue, 22 Sep 2015 22:04:52 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id A8FAB15344D; Wed, 23 Sep 2015 00:04:39 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9pshHBITbtdI; Wed, 23 Sep 2015 00:04:20 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:9d1a:d16e:7a5b:d559] (unknown [IPv6:2001:4cb8:3:1:9d1a:d16e:7a5b:d559]) by smtp.digiware.nl (Postfix) with ESMTPA id D85C9153431; Wed, 23 Sep 2015 00:04:20 +0200 (CEST) Subject: Re: [HAST] ZFS, many disks, write order To: Mark Felder , InterNetX - Juergen Gotteswinter , freebsd-fs@freebsd.org References: <560102BD.6040703@internetx.com> <1442953467.1530841.390840385.067EC1E4@webmail.messagingengine.com> From: Willem Jan Withagen Message-ID: <5601D063.4020104@digiware.nl> Date: Wed, 23 Sep 2015 00:04:19 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <1442953467.1530841.390840385.067EC1E4@webmail.messagingengine.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 22:04:52 -0000 On 22-9-2015 22:24, Mark Felder wrote: > > > On Tue, Sep 22, 2015, at 02:26, InterNetX - Juergen Gotteswinter wrote: >> One of the great Hast Features is that it Hides Disk Failures from ZFS, >> this has been already discussed on this List a few Months ago. >> >> Single ZIL Flash Drive is also not such a great Idea. I whould suggest >> to start from scratch with "ZFS Best Practices". >> > > A single ZIL is not a single point of failure for a few years now. Most > people do not have a ton of data going through the ZIL and if it dies > you only lose those transactions in flight, not your entire pool. Well if you are talking about HA, then it is not quite in order to lose some transactions. Especially if they originate from synchronous writes that are reported committed to the issuer. So why not have 2 in mirror, either both local, or again in local/remote setup. For caches I'd not use this model, since caches are really not critical. If they die, the data is just re-fetched from disk. --WjW From owner-freebsd-fs@freebsd.org Tue Sep 22 23:26:29 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A2EA0A07ED0 for ; Tue, 22 Sep 2015 23:26:29 +0000 (UTC) (envelope-from tjg@ucsc.edu) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3F4331A52 for ; Tue, 22 Sep 2015 23:26:29 +0000 (UTC) (envelope-from tjg@ucsc.edu) Received: by wicgb1 with SMTP id gb1so181968733wic.1 for ; Tue, 22 Sep 2015 16:26:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=mime-version:date:message-id:subject:from:to:content-type; bh=h67CcowrLuqBohZscumxtKt2JbkGr+0AeD/UD8ZGXok=; b=CjObWXzwqOrGZ2wYvTCHAhxYFo1ryCVkgZfjtdBoJGSt0VV50f812x2pl1ZV4RtMH6 76D3VseO7OjeH+IV7MyCkIQUPfwvIaCaPwI8BvGQQOe6FAKsJ8QAWYXUG9WP5WOJ1ed6 CnPMSAz9MFvap7mPruTTb7+26j82l1qWrVBPg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=h67CcowrLuqBohZscumxtKt2JbkGr+0AeD/UD8ZGXok=; b=L38TDhih0u95jUhcEJvAB9SxsTDkeiw6kw+cwtrYIkyoVwWtMXNj+rETExlNgf9cLz 0caE4cG1UNOgod1+xegYv6uhtbkOKDQSCVQIMeS/0hEfBu0YFaAFEcTqT73Mo3NKpB0O aGlg4lfVAK5t9+KdgMsWxwis4pnzYeaCVv41276ZUAdQgdf6mQHCoOfY9EVK40jQxuIe LqQXAtfYCOuEyn9lgWH/vSds+HTPXVEFwd/pQan/ksZmolL+RlU4T92pr01zootjT/2m TYoIAPu1mQVXNCJzazhXg/4o9A1yuZxDR+drMwcuQHCiqZz12BLHpWDFAe+mu4udP8At ON6A== X-Gm-Message-State: ALoCoQkuxc31TVlAbLBvL5MMbRC1VUlTqxkz4g5zn2niQxc7UeHq31gzJVW7itLhnvIFOr6RPIfe MIME-Version: 1.0 X-Received: by 10.180.8.132 with SMTP id r4mr347833wia.70.1442964387556; Tue, 22 Sep 2015 16:26:27 -0700 (PDT) Received: by 10.194.92.209 with HTTP; Tue, 22 Sep 2015 16:26:27 -0700 (PDT) Date: Tue, 22 Sep 2015 16:26:27 -0700 Message-ID: Subject: Switching from MFI to MRSAS From: Tim Gustafson To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 23:26:29 -0000 As I mentioned in my previous post last week, I have a new Dell R630 server that uses a Perc controller, and I have installed 4 SSD drives and configured the PERC controller in pass-through mode. However, the PERC controller and its drives are being handled by the "mfi" driver, which apparently does not support TRIM. I have been reading up on all this, and I see that there is also an "mrsas" driver that (according to the documentation) appears to be compatible with my PERC controller, which is identified as follows: mfi0: port 0x2000-0x20ff mem 0x92000000-0x9200ffff,0x91f00000-0x91ffffff irq 26 at device 0.0 on pci2 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 mfi0: FW MaxCmds = 240, limiting to 128 mfi0: MaxCmd = 240, Drv MaxCmd = 128, MaxSgl = 70, state = 0xb73c00f0 I see that there's a tunable for /boot/device.hints that I can set to prefer mrsas over mfi: hw.mfi.mrsas_enable="1" and then add this to /boot/loader.conf: mrsas_load="yes" So, my question is: if I add those options and then reboot, will my drives magically re-appear as /dev/da* devices? Will the data on them still be accessible as-is, or would I need to re-install the OS to get this to go? Would switching to this other driver enable the TRIM command, as the devices will now be /dev/da* rather than /dev/mfisyspd*? Or am I just being naive here? :) -- Tim Gustafson tjg@ucsc.edu 831-459-5354 Baskin Engineering, Room 313A From owner-freebsd-fs@freebsd.org Wed Sep 23 05:43:53 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 783DAA07BA4 for ; Wed, 23 Sep 2015 05:43:53 +0000 (UTC) (envelope-from michael@fuckner.net) Received: from mo6-p00-ob.smtp.rzone.de (mo6-p00-ob.smtp.rzone.de [IPv6:2a01:238:20a:202:5300::11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.smtp.rzone.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1608412D0 for ; Wed, 23 Sep 2015 05:43:52 +0000 (UTC) (envelope-from michael@fuckner.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1442987029; l=867; s=domk; d=fuckner.net; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version: Date:From:References:To:Subject; bh=Qzq2rZevPRQfaWpKMtrwR0ZWUcp42/2MxbOy8lUZLw8=; b=pYpSYDB0gwbAC24zuVmTJIAUWm9+zX/o4XKhMtIuHokmDg/oy39sH6xtt6eMl2MU+54 gtwGrlXSJc51Kit/VjekEZqHvI2jDnrduDH/AT2XxiyMxpI4EWXfXKfJIQf4iiTIE+hCp 4BrqjdrVnx1Tteyz/yyHfPeHqySe4KBo9ag= X-RZG-AUTH: :IWUHfUGtd9+6EujMWHx57N4dWae4bmTL/JIGbzkGUoozgknstV9BEzWRmW1WTNAldMQ9WrX0XNX2YqcutSnrNOxNOjrGw3F0LA== X-RZG-CLASS-ID: mo00 Received: from [IPv6:2a02:2028:53f:9001:1c46:b4db:2412:9ab7] (some-ipv6-address.wtnet.de [IPv6:2a02:2028:53f:9001:1c46:b4db:2412:9ab7]) by smtp.strato.de (RZmta 37.12 AUTH) with ESMTPSA id 204cbdr8N5hm0IN (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (curve secp521r1 with 521 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate); Wed, 23 Sep 2015 07:43:48 +0200 (CEST) Subject: Re: Switching from MFI to MRSAS To: Tim Gustafson , freebsd-fs@freebsd.org References: From: Michael Fuckner Message-ID: <56023C15.7080007@fuckner.net> Date: Wed, 23 Sep 2015 07:43:49 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 05:43:53 -0000 On 9/23/2015 1:26 AM, Tim Gustafson wrote: > As I mentioned in my previous post last week, I have a new Dell R630 > server that uses a Perc controller, and I have installed 4 SSD drives > and configured the PERC controller in pass-through mode. However, the > PERC controller and its drives are being handled by the "mfi" driver, > which apparently does not support TRIM. > > I have been reading up on all this, and I see that there is also an > "mrsas" driver that (according to the documentation) appears to be > compatible with my PERC controller, which is identified as follows: > Hi, mfi is the old driver for 3GBit and 6GBit (Single Core) Raid Cards (102108 based) mrsas is the newer driver for 6GBit (Dual Core) and 12GBit Cards (2208/ 3008 based) Different driver for different LSI Chipsets Sorry to disappoint you, regards, Michael From owner-freebsd-fs@freebsd.org Wed Sep 23 06:36:55 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C5BF7A0763F for ; Wed, 23 Sep 2015 06:36:55 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from Exch2-3.slu.se (pop.slu.se [77.235.224.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 590A4162D for ; Wed, 23 Sep 2015 06:36:54 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by Exch2-3.slu.se (77.235.224.123) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Wed, 23 Sep 2015 08:21:41 +0200 Received: from exch2-4.slu.se ([fe80::bc24:95e8:1e1e:f3ae]) by exch2-4.slu.se ([fe80::bc24:95e8:1e1e:f3ae%22]) with mapi id 15.00.1104.000; Wed, 23 Sep 2015 08:21:41 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Chris Stankevitz CC: Warren Block , FreeBSD FS , Matt Churchyard Subject: Re: Name/label/id metadata: how do I make it go away Thread-Topic: Name/label/id metadata: how do I make it go away Thread-Index: AQHQ9Jy8yHBRX2DQ2EWvFLWvzTjdJZ5Hxl8AgAAproCAACLFgIAAt7wAgAAMioCAABHSAIAAC8qAgACQwgA= Date: Wed, 23 Sep 2015 06:21:40 +0000 Message-ID: <1442989300.5271.9.camel@data-b104.adm.slu.se> References: <56004C68.4020904@stankevitz.com> <5600F0DF.8000805@stankevitz.com> <5601A82A.7040304@stankevitz.com> <5601B2AF.7040306@stankevitz.com> <5601CB85.8070400@stankevitz.com> In-Reply-To: <5601CB85.8070400@stankevitz.com> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [77.235.228.63] Content-Type: text/plain; charset="utf-8" Content-ID: <92CF5E89249A2B4F8766BF3CD07035B3@ad.slu.se> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 06:36:56 -0000 dGlzIDIwMTUtMDktMjIga2xvY2thbiAxNDo0MyAtMDcwMCBza3JldiBDaHJpcyBTdGFua2V2aXR6 Og0KPiBPbiA5LzIyLzE1IDI6MDEgUE0sIFdhcnJlbiBCbG9jayB3cm90ZToNCj4gPj4gQWNjb3Jk aW5nIHRvICdnZW9tIGRpc2sgbGlzdCcsIGVhY2ggb2YgbXkgMjIgZHJpdmVzIGhhcyBhbiAiaWRl bnQiLA0KPiA+PiBidXQgb25seSBzb21lIG9mIHRoZW0gaGF2ZSAibGFiZWxzIi4NCj4gPj4NCj4g Pj4gImlkZW50IiBpcyB0dXJuZWQgaW50byBhICJsYWJlbCIgdmlhIGEgcHJvY2VzcyBjYWxsZWQg InRhc3RpbmciOg0KPiA+PiAgc3lzL2dlb20vbGFiZWwvZ19sYWJlbF9kaXNrX2lkZW50LmMNCj4g Pj4gIHN5cy9nZW9tL2xhYmVsL2dfbGFiZWwuYw0KPiA+Pg0KPiA+PiBBbHRob3VnaCBpdCBpcyBu b3QgeWV0IGNsZWFyIHRvIG1lIHdoeSBhcHBhcmVudGx5IG9ubHkgc29tZSBvZiBteQ0KPiA+PiBk aXNrcyBhcmUgInRhc3RlZCIuDQo+ID4NCj4gPiBMYWJlbHMgb24gZGV2aWNlcyB0aGF0IGFyZSBt b3VudGVkIGFyZSAid2l0aGVyZWQiIGFuZCBkaXNhcHBlYXIuDQo+IA0KPiBUaGFuayB5b3UuDQo+ IA0KPiBJbmRlZWQgd2hlbiBJIGV4cG9ydCB0aGUgcG9vbCwgYWxsIG9mIHRoZSBkcml2ZXMgYmVj b21lIGxpc3RlZCBpbiANCj4gL2Rldi9kaXNraWQuICBXaGVuIEkgcmUtaW1wb3J0IHRoZSBwb29s LCBzb21lIG9mIHRoZSBkcml2ZXMgIndpdGhlciIgYW5kIA0KPiBhcmUgcmVtb3ZlZCBmcm9tIC9k ZXYvZGlza2lkLiAgVGhlc2Ugd2l0aGVyZWQgZHJpdmVzIGFwcGVhciBhcyBkYVggaW4gDQo+ICJ6 cG9vbCBzdGF0dXMiLiAgVGhlIGRyaXZlcyB0aGF0IGRvIG5vdCB3aXRoZXIgcmVtYWluIGluIC9k ZXYvZGlza2lkIGFuZCANCj4gYXBwZWFyIGluICJ6cG9vbCBzdGF0dXMiIHdpdGggdGhlaXIgL2Rl di9kaXNraWQvWCBuYW1lcy4NCj4gDQo+IFE6IFdoeSBkbyBteSBpZGVudGljYWwgenBvb2wgZHJp dmVzIGJlaGF2ZSBkaWZmZXJlbnRseT8NCj4gDQo+IFBsZWFzZSBsZXQgbWUga25vdyBpZiBteSBo eXBvdGhlc2lzIGlzIHBsYXVzaWJsZToNCj4gDQo+IFpGUyBzZWVzIGJvdGggdGhlIC9kZXYvZGFY ICJjb25zdW1lcnMiIGFuZCB0aGUgL2Rldi9kaXNraWQvWCANCj4gInByb3ZpZGVycyIuICBaRlMg aXMgcmFuZG9tbHkgc2VsZWN0aW5nIHdoaWNoIHRvIHVzZSB0byBpbXBvcnQgdGhlIHBvb2wuIA0K PiAgIFNvbWV0aW1lcyBpdCBwaWNrcyAiY29uc3VtZXJzIiBhbmQgc29tZXRpbWVzICJwcm92aWRl cnMiLg0KPiANCj4gV2hlbiBaRlMgcGlja3MgYSBjb25zdW1lciwgdGhlIHByb3ZpZGVyIGlzIHdp dGhlcmVkIGF3YXkuICBXaGVuIFpGUyANCj4gcGlja3MgYSBwcm92aWRlciwgdGhlIHByb3ZpZGVy IHJlbWFpbnMuDQo+IA0KPiBBbmQgaWYgSSB3YW50IHRvIGRpZyBkZWVwZXIgaW50byByb290IGNh dXNlIEkgY2FuIGFzayBaRlMgIndoeSBkbyB5b3UgDQo+IHNvbWV0aW1lcyBzZWxlY3QgZnJvbSB0 aGUgY29uc3VtZXIgY29sbGVjdGlvbiBhbmQgc29tZXRpbWVzIGZyb20gdGhlIA0KPiBwcm92aWRl ciBjb2xsZWN0aW9uIHdoZW4gcHV0dGluZyBhIHBvb2wgdG9nZXRoZXIiLiAgT3IgaWYgSSBkb24n dCB3YW50IA0KPiB0byBkaWcgZGVlcGVyIEkgY2FuICJkZWFsIHdpdGggaXQiIG9yIEkgY2FuIGRp c2FibGUgZGlza2lkIHVzaW5nIA0KPiBrZXJuLmdlb20ubGFiZWwuZGlza19pZGVudC5lbmFibGUN Cg0KQ2FuwrR0IHNheSBtdWNoIGFib3V0IHRoZSByZXN0IGJ1dCB5ZWFoLCBpZiB5b3Ugd2FudCB0 aGUgZGlza2lkJ3MgdG8gZ28NCmF3YXkgZ2xvYmFsbHksIHRoYXTCtHMgeW91ciB0aWNrZXQuIElm IHlvdcK0cmUgb25seSBjb25jZXJuZWQgYWJvdXQgJ3pwb29sDQpzdGF0dXMnIGxvb2tpbmcgd2ll cmQsIHlvdSBjYW4gZXhwb3J0IHRoZSBwb29sIGFuZCB0aGVuIGltcG9ydCBpdCBhZ2Fpbg0Kd2hp bGUgc3BlY2lmeWluZyB3aGVyZSBaRlMgc2hvdWxkIGxvb2sgZm9yIHRoZSBwcm92aWRlcnM6DQoN CiMgenBvb2wgaW1wb3J0IC1kIC9kZXYgPHBvb2w+DQoNCi9LDQoNCj4gDQo+IFRoYW5rIHlvdSBh Z2FpbiwNCj4gDQo+IENocmlzDQo+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fDQo+IGZyZWVic2QtZnNAZnJlZWJzZC5vcmcgbWFpbGluZyBsaXN0DQo+IGh0 dHBzOi8vbGlzdHMuZnJlZWJzZC5vcmcvbWFpbG1hbi9saXN0aW5mby9mcmVlYnNkLWZzDQo+IFRv IHVuc3Vic2NyaWJlLCBzZW5kIGFueSBtYWlsIHRvICJmcmVlYnNkLWZzLXVuc3Vic2NyaWJlQGZy ZWVic2Qub3JnIg0KDQo= From owner-freebsd-fs@freebsd.org Wed Sep 23 07:06:02 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDEFCA06449 for ; Wed, 23 Sep 2015 07:06:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B9C45121F for ; Wed, 23 Sep 2015 07:06:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8N762Ju001307 for ; Wed, 23 Sep 2015 07:06:02 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 202607] [panic] Poudriere umounting file systems causes 'solaris assert: avl_is_empty(&dn->dn_dbufs)' panic Date: Wed, 23 Sep 2015 07:06:02 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 07:06:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202607 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Wed Sep 23 09:33:45 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C506A06B33 for ; Wed, 23 Sep 2015 09:33:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7905617F7 for ; Wed, 23 Sep 2015 09:33:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8N9XjH9040708 for ; Wed, 23 Sep 2015 09:33:45 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 202607] [panic] Poudriere umounting file systems causes 'solaris assert: avl_is_empty(&dn->dn_dbufs)' panic Date: Wed, 23 Sep 2015 09:33:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 09:33:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202607 --- Comment #18 from Andriy Gapon --- Another note: if I exclude DMU_OT_BPOBJ dnodes from the check, then the assertion is not hit. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Wed Sep 23 15:55:55 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39E47A07091 for ; Wed, 23 Sep 2015 15:55:55 +0000 (UTC) (envelope-from tjg@ucsc.edu) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C5BD812C3 for ; Wed, 23 Sep 2015 15:55:54 +0000 (UTC) (envelope-from tjg@ucsc.edu) Received: by wicge5 with SMTP id ge5so214035125wic.0 for ; Wed, 23 Sep 2015 08:55:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sUqOpj5XGH35951jGTPZOiX4nGqp10sk13Caa05zrKA=; b=UVKHxpStLGxhu/Fm0/dGhazCmhlqdFKQon29/lx4OGs3JpwzBtmHHo1W9G+bzahl5z 5WkmeQ0a2YqWIi7VKXESHXpKTG/WdmITB2jlOsFiOS3QS6iHbBwJvxFwoSTvyrD52/L6 DLPf+LGwHeVzhTqOkyI4vGTetZk2+6E0n5ZzE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=sUqOpj5XGH35951jGTPZOiX4nGqp10sk13Caa05zrKA=; b=Mw7iwBfWYfaLLY8Oj+cLTgbnQXcTn04GKAiZv483+fGk0dLBvXOfYAZBLcxLL+37Ko nQdlD9XhtzOi5/LXAMQW+ISpwZ/wNzs9wRQ8hEHJFUyGxW1un5PaxfCj2B1bZEpvFq9G 0C0EKOQxOXmu3Pkv1f82c7prfAqGFhnnWzT+4o5grRwPYj5ZvSqm3fhdjbN1zy+4uf3K MFceegQe/fkXZBsKP6rCxB2CVfil5bFqIEtFCtrdNptHqf0ToL4v9/q/OtNUxzujXccZ QZfXBCbs4JAArotpdPxI7Un624vuvYuCadggzOmVghBOb8PdgmFbmqYrWjuy3bLIdARw XS+Q== X-Gm-Message-State: ALoCoQnkvZDDGNtS8yX7Rolu/AjFYgTn6oWBnKrKNvL2aCz0ysFd+kWDS9uD7w+w5TAs5FeLnS8s MIME-Version: 1.0 X-Received: by 10.180.198.180 with SMTP id jd20mr5150042wic.70.1443023753119; Wed, 23 Sep 2015 08:55:53 -0700 (PDT) Received: by 10.194.92.209 with HTTP; Wed, 23 Sep 2015 08:55:53 -0700 (PDT) In-Reply-To: <56023C15.7080007@fuckner.net> References: <56023C15.7080007@fuckner.net> Date: Wed, 23 Sep 2015 08:55:53 -0700 Message-ID: Subject: Re: Switching from MFI to MRSAS From: Tim Gustafson To: Michael Fuckner Cc: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 15:55:55 -0000 > Different driver for different LSI Chipsets That's not what the man page says. The man page says: "Older MegaRAID controllers are supported by mfi(4) and will not work with mrsas, but both the mfi(4) and mrsas drivers can detect and manage the LSI MegaRAID SAS 2208/2308/3008/3108 series of controllers." And goes on to say: "The device.hints(5) option is provided to tune the mrsas driver's behavior for LSI MegaRAID SAS 2208/2308/3008/3108 controllers. By default, the mfi(4) driver will detect these controllers. See the PRIORITY section to know more about driver priority for MR-Fusion devices. mrsas will provide a priority of (-30) (between BUS_PROBE_DEFAULT and BUS_PROBE_LOW_PRIORITY) at probe call for device id's 0x005B, 0x005D, and 0x005F so that mrsas does not take control of these devices without user intervention." The FreeBSD hardware support page specifically says that the "mrsas" driver will handle the Dell PERC H330 controller (which is what I have): https://www.freebsd.org/relnotes/CURRENT/hardware/support.html So that's what I'm asking about. I have a newer card, which according to the man page is supported by the mrsas driver. So my question is, if I "flip the switch" so that the mrsas driver takes over, will everything "just" work, or will I need to re-format my disks because the "mfi" driver configures the disks in a way that is somehow incompatible with the "mrsas" driver. -- Tim Gustafson tjg@ucsc.edu 831-459-5354 Baskin Engineering, Room 313A From owner-freebsd-fs@freebsd.org Wed Sep 23 16:16:42 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8EAA2A07A60 for ; Wed, 23 Sep 2015 16:16:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7BBE7124B for ; Wed, 23 Sep 2015 16:16:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8NGGg2k059375 for ; Wed, 23 Sep 2015 16:16:42 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 198457] zfs acl lost after zfs send-receive. Kernel panic Date: Wed, 23 Sep 2015 16:16:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: trasz@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 16:16:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198457 --- Comment #3 from Edward Tomasz Napierala --- So, from my reading, it panics, because for some reason it cannot gracefully handle error returned by zfs_acl_node_read(). Why? No idea. Note however, that this is code that's not specific to FreeBSD - it might be a good idea to consult Nexenta or ZoL folks. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Wed Sep 23 16:38:22 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2BA9FA064D9 for ; Wed, 23 Sep 2015 16:38:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 18B661ED6 for ; Wed, 23 Sep 2015 16:38:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8NGcLYp096493 for ; Wed, 23 Sep 2015 16:38:21 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 198457] zfs acl lost after zfs send-receive. Kernel panic Date: Wed, 23 Sep 2015 16:38:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: o.bende@gmail.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 16:38:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198457 --- Comment #4 from o.bende@gmail.com --- Thanks for your Comment. I think the main problem is not the handling of the error, but that there is an error! I can (probably) understand, why the system should panic because of such a file-system issue - but why is there this issue? Summary: Why are the ACLs broken after replication? And another question: Why do you think we should contact ZoL? I always thought, ZoL and FreeBSD ZFS are two different systems?! Actually, I did not choose ZoL because it is not really defined production-ready - but FreeBSD ZFS is. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Wed Sep 23 16:46:34 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6605FA06BB4 for ; Wed, 23 Sep 2015 16:46:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 525E9177A for ; Wed, 23 Sep 2015 16:46:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8NGkYE5014314 for ; Wed, 23 Sep 2015 16:46:34 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 202607] [panic] Poudriere umounting file systems causes 'solaris assert: avl_is_empty(&dn->dn_dbufs)' panic Date: Wed, 23 Sep 2015 16:46:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: gibbs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 16:46:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202607 --- Comment #19 from Justin T. Gibbs --- Thinking about this some more, I think it may be possible for a dbuf for an older version of a dnode to be reused if it hasn't been evicted before the object id is reused. For most objects (e.g. files), there are no dangling references due to dbuf user eviction. But this can happen for datasets, and we can't know the future type of the object that will inherit this object id. My preference would be some solution that doesn't cause TXG syncing to block. All we need to do is to mark all the dbufs as no longer cacheable in dnode_evict_dbufs() during the pass that we try to evict them. This would allow dbuf_rele_and_unlock() to determine whether eviction processing is necessary without taking the dnode lock. It looks like dmu_buf_impl_t has internal padding due to the order of its fields, so this state can probably be added without any space penalty. If you do this, the code to forcibly evict the bonus buffer in dbuf_rele_and_unlock() should be simplified to use the same logic. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Wed Sep 23 16:57:13 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 23987A07151 for ; Wed, 23 Sep 2015 16:57:13 +0000 (UTC) (envelope-from michael@fuckner.net) Received: from mo6-p00-ob.smtp.rzone.de (mo6-p00-ob.smtp.rzone.de [IPv6:2a01:238:20a:202:5300::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.smtp.rzone.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B4BAC1E00 for ; Wed, 23 Sep 2015 16:57:12 +0000 (UTC) (envelope-from michael@fuckner.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1443027430; l=995; s=domk; d=fuckner.net; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version: Date:From:Cc:References:To:Subject; bh=FSrOtI2MPUjk0H+ASH0BgzYlS3FHlxwUNronDm+IE48=; b=gtskZWUSmJcIDj7B+hZxGNuXJ2rQO22h9IXmJrxMroq1riGYNK3SFGjWQlb93ZYBixO GEYbdP8UeoZ6TvXriSuMyBI+rGnZLTGzXbG2iCUEM0U6hKVJ1vRt3glVT/JE7UCx3d70V rBjggCl6KUhmEP77elnXc7n5rJSATV92iUY= X-RZG-AUTH: :IWUHfUGtd9+6EujMWHx57N4dWae4bmTL/JIGbzkGUoozgknstV9BEzWRmW1WTNAldMQ9WrX8XNG6HyWAdDHhg/bmxNRKdgEB3w== X-RZG-CLASS-ID: mo00 Received: from [IPv6:2a02:2028:53f:9001:9c0d:d407:b785:dd54] (some-ipv6-address.wtnet.de [IPv6:2a02:2028:53f:9001:9c0d:d407:b785:dd54]) by smtp.strato.de (RZmta 37.12 AUTH) with ESMTPSA id e042e3r8NGv9JEP (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (curve secp521r1 with 521 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate); Wed, 23 Sep 2015 18:57:09 +0200 (CEST) Subject: Re: Switching from MFI to MRSAS To: Tim Gustafson References: <56023C15.7080007@fuckner.net> Cc: freebsd-fs@freebsd.org From: Michael Fuckner Message-ID: <5602D9E8.2090903@fuckner.net> Date: Wed, 23 Sep 2015 18:57:12 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 16:57:13 -0000 On 9/23/2015 5:55 PM, Tim Gustafson wrote: >> Different driver for different LSI Chipsets > > That's not what the man page says. The man page says: > > "Older MegaRAID controllers are supported by mfi(4) and will not work > with mrsas, but both the mfi(4) and mrsas drivers can detect and > manage the LSI MegaRAID SAS 2208/2308/3008/3108 series of > controllers." > OK, haven't done this before, perhaps this works, but... you are replacing a raid driver with another raid driver, so probably TRIM will not work either. > So that's what I'm asking about. I have a newer card, which according > to the man page is supported by the mrsas driver. So my question is, > if I "flip the switch" so that the mrsas driver takes over, will > everything "just" work, or will I need to re-format my disks because > the "mfi" driver configures the disks in a way that is somehow > incompatible with the "mrsas" driver. the on disk format should be compatible. Good luck, Michael! From owner-freebsd-fs@freebsd.org Wed Sep 23 16:59:57 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 978E6A07342 for ; Wed, 23 Sep 2015 16:59:57 +0000 (UTC) (envelope-from tjg@ucsc.edu) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C308105D for ; Wed, 23 Sep 2015 16:59:57 +0000 (UTC) (envelope-from tjg@ucsc.edu) Received: by wicfx3 with SMTP id fx3so78681250wic.0 for ; Wed, 23 Sep 2015 09:59:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+NXh8Co4MtTAciFu5Ru5F9PV0CLke/pjpUcVbYDrdq0=; b=df8wPhsJa+n0wdMXPicccI3p1LB9jcyzvOubwVK+eDq2S3YNCnnF4Z0nDGvHeFYVEP 4LgUaJ32CJZv3q57WPkTsY+nINjGtyacTr3998pDHNbNol5acl9royL7RF2KEwN1m1oB BjHeOEbyUwqnH5DcKyxq8TsXGJAactj6rWggc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=+NXh8Co4MtTAciFu5Ru5F9PV0CLke/pjpUcVbYDrdq0=; b=g8AzBwiOQ61RMTDNRGhsla5e7vsQuEz1UiC7hQOlkmqiKAfIx2BDuZuQQlgAdSUClB tkJKkN6gbETiaogoyE25PBeN8ZcKW7BAcbsVhD47MD7PQE8hsJftZTGIZYSwFvKdvUis sHYmT/EUKRU4Nr4sOsJeMZ7DRkDK1hx/sW611M0EA/xe+8xNjIIftVkO5td03yNtC8KU 0ylIOVw/BR1cSyChvWuSEQUVIkmUxjZrCdfnq0mnqR2+4D+cW0S9enEhBXQqDVbHknz/ A46BkH+21TSCpQimnVOtud6FA8VPuCSCjKH5tieiD4ix7+aCVFTE+OleSp/ODSnu72G8 1vyw== X-Gm-Message-State: ALoCoQl5Q4alejywhP9qRyisFGkuApawCemW9pSxvoxZWA2zOKhE6jZZvYIcPD+tYiKg94VevSpY MIME-Version: 1.0 X-Received: by 10.180.187.141 with SMTP id fs13mr5467626wic.13.1443027594697; Wed, 23 Sep 2015 09:59:54 -0700 (PDT) Received: by 10.194.92.209 with HTTP; Wed, 23 Sep 2015 09:59:54 -0700 (PDT) In-Reply-To: <5602D9E8.2090903@fuckner.net> References: <56023C15.7080007@fuckner.net> <5602D9E8.2090903@fuckner.net> Date: Wed, 23 Sep 2015 09:59:54 -0700 Message-ID: Subject: Re: Switching from MFI to MRSAS From: Tim Gustafson To: Michael Fuckner Cc: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 16:59:57 -0000 > OK, haven't done this before, perhaps this works, but... you are replacing a > raid driver with another raid driver, so probably TRIM will not work either. The mrsas driver man page says that it reports the disks as "da0", "da1" and so on, and the man pages about TRIM support all say that it works with "ada" and "da" drives. So, I was hoping that the change in driver technology would enable the TRIM command to work. I suppose I could update device.hints and loader.conf as describe, reboot, and if the system doesn't come back up, just boot off a thumb drive and re-edit those files back to their defaults... -- Tim Gustafson tjg@ucsc.edu 831-459-5354 Baskin Engineering, Room 313A From owner-freebsd-fs@freebsd.org Wed Sep 23 17:06:03 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D08C5A07822 for ; Wed, 23 Sep 2015 17:06:03 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 95A8818EF for ; Wed, 23 Sep 2015 17:06:03 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by obbda8 with SMTP id da8so38595239obb.1 for ; Wed, 23 Sep 2015 10:06:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=xBfsknFn4NZ1M6kLKZ2F5iYJaRekQkJdoLhLv6D4q/E=; b=pddfHSybDrbBHbgcrGwuBNQ5vhGnMO0DeODQEUFv4Mcwu8oI4/anx8tcsgLZsW/rPA ph8vbipqqhYCbc+RESxWKmvSBEU9r11Ru6opjJc2ZxF1H7r6AcDqHkTuKbOG7uVh5JeR gYaO7U7qz5eW9k0fEc8O5xAIoPBg6nCVimVwdtrR3LqeqvI3qRQ228QRFBHMqyKKo2pA opavx95RI6e2+/KZJVnpVzAvN7ng7VVsENGRlCq5KDiD0DEt4IgV8GubXzK0tDzF0uaD R+axJlSj3aCelGsHdUt9X4nhOrxQETgH5haKva3pBU5jn19XkUean4ue4LLM/B/UCNZu xZ2Q== MIME-Version: 1.0 X-Received: by 10.182.47.202 with SMTP id f10mr19354972obn.56.1443027962630; Wed, 23 Sep 2015 10:06:02 -0700 (PDT) Received: by 10.76.68.38 with HTTP; Wed, 23 Sep 2015 10:06:02 -0700 (PDT) In-Reply-To: References: <56023C15.7080007@fuckner.net> <5602D9E8.2090903@fuckner.net> Date: Wed, 23 Sep 2015 10:06:02 -0700 Message-ID: Subject: Re: Switching from MFI to MRSAS From: Freddie Cash To: Tim Gustafson Cc: Michael Fuckner , FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 17:06:04 -0000 On Wed, Sep 23, 2015 at 9:59 AM, Tim Gustafson wrote: > > OK, haven't done this before, perhaps this works, but... you are > replacing a > > raid driver with another raid driver, so probably TRIM will not work > either. > > The mrsas driver man page says that it reports the disks as "da0", > "da1" and so on, and the man pages about TRIM support all say that it > works with "ada" and "da" drives. So, I was hoping that the change in > driver technology would enable the TRIM command to work. > > I suppose I could update device.hints and loader.conf as describe, > reboot, and if the system doesn't come back up, just boot off a thumb > drive and re-edit those files back to their defaults... =E2=80=8BOr, reboot the system, and drop to the loader prompt. Then manual= ly type in those loader.conf entries at the loader prompt. And continue the boot process. If it fails, then just reboot, and everything will come up normally. If it succeeds, then you can edit /boot/loader.conf to make the change permanent. You'll want to boot to single-user mode as well, as /etc/fstab will point to the wrong devices (unless you are using some kind of generic labels for the entries instead of the physical device node names). In single-user mode, you can update /etc/fstab and /boot/loader.conf as needed if the new driver works.=E2=80=8B --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@freebsd.org Wed Sep 23 17:48:30 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B06DA06D0E for ; Wed, 23 Sep 2015 17:48:30 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B1DCF1C35 for ; Wed, 23 Sep 2015 17:48:29 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from liminal.local (liminal.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3636:3bff:fed4:b0d6]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.15.2/8.15.2) with ESMTPSA id t8NHmM63068472 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 23 Sep 2015 18:48:22 +0100 (BST) (envelope-from matthew@FreeBSD.org) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org DKIM-Filter: OpenDKIM Filter v2.10.3 smtp.infracaninophile.co.uk t8NHmM63068472 Authentication-Results: smtp.infracaninophile.co.uk/t8NHmM63068472; dkim=none; dkim-atps=neutral X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host liminal.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3636:3bff:fed4:b0d6] claimed to be liminal.local Subject: Re: Switching from MFI to MRSAS To: freebsd-fs@freebsd.org References: <56023C15.7080007@fuckner.net> From: Matthew Seaman X-Enigmail-Draft-Status: N1110 Message-ID: <5602E5DC.9000009@FreeBSD.org> Date: Wed, 23 Sep 2015 18:48:12 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ph7gpDH09hwMSRS3Wg0R6w6bCG9KecFBH" X-Virus-Scanned: clamav-milter 0.98.7 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 17:48:30 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ph7gpDH09hwMSRS3Wg0R6w6bCG9KecFBH Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 23/09/2015 16:55, Tim Gustafson wrote: > The FreeBSD hardware support page specifically says that the "mrsas" > driver will handle the Dell PERC H330 controller (which is what I > have): >=20 > https://www.freebsd.org/relnotes/CURRENT/hardware/support.html >=20 > So that's what I'm asking about. I have a newer card, which according > to the man page is supported by the mrsas driver. So my question is, > if I "flip the switch" so that the mrsas driver takes over, will > everything "just" work, or will I need to re-format my disks because > the "mfi" driver configures the disks in a way that is somehow > incompatible with the "mrsas" driver. We've a bunch of Dell machines of the H730 cards, which are (I think) the same chipset but with various additional bits like a BBU and nvram and so forth. Yes, if mfi(8) detects the controller but doesn't work (or doesn't perform) then the mrsas driver is the thing to use. It works fine as a RAID controller, but there is no equivalent of mfiutil(8) for these cards= =2E Trim support -- or the lack of it -- is apparently a feature of the firmware on Dell-branded SSD drives. Cheers, Matthew --ph7gpDH09hwMSRS3Wg0R6w6bCG9KecFBH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2 iQJ8BAEBCgBmBQJWAuXiXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkAT/fIQAJ0gS6gRIslRqZIN/53o0P6y XEBXuTdSloskPGtzca7hYXSVPq6PkmEOdY+kGyx5kjbgdOkwAKXnm3Jy7hD9/bMm xBJEke3DcxXKu7J4fXFTjswZLir7p7Oa6DuY0A/37PNF3On/OGwRhlmWtBD/prZ7 B9OGQh2AD+lw0YpQDIyMLzWnDI/TF7szZBXotcMFQ8QhOCO5ES0ZnjsP5rt4Gz7G MfJGlkqyosPbj/LCgwMCa6OlBpR+L9TfdBlfm5PzsI+mXmoEWWnk8E2B6Q9PPDAt PZDR1BdYR67QddIWWqx2ADPAR+PMcOu/BK3Ya9IzQ/rei0sgD/jlaXaag5Yo0P1g 9hT1fh1n4gRZFYy27VKQwvtL2wa/wvk680elFKhNzh+39ZT78Pj/XXcbnZ0SVnRn /HzYzALrDt6kMu6eKUJp3oaG8HdmzaV4GQGxBmwaO6xcDWZb2bq8N0EDBsTXv9El vKEbOGopn8mRylUPvtV3u8ioxuB6h2ToAwhTv2/YDPl6Tm9Y6B4tyISazqYLoQHW lki2AXSSRMxzb0zNKUL6PC7QQf1QwIw+UbNjc2Z/zqGh/zBwVdaF3iuY1xuxxnnH 5A3L8d6fmVjYFHxxwwfwSupxn6n1pdjb38fZdFF4pu+TJ0gVPUlBxNE4IILY0BGp xqTs+1IPFHhzgfFN0kI1 =NWXq -----END PGP SIGNATURE----- --ph7gpDH09hwMSRS3Wg0R6w6bCG9KecFBH-- From owner-freebsd-fs@freebsd.org Wed Sep 23 19:03:52 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9CF23A074A0 for ; Wed, 23 Sep 2015 19:03:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8A15711DC for ; Wed, 23 Sep 2015 19:03:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8NJ3q9b060711 for ; Wed, 23 Sep 2015 19:03:52 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 202607] [panic] Poudriere umounting file systems causes 'solaris assert: avl_is_empty(&dn->dn_dbufs)' panic Date: Wed, 23 Sep 2015 19:03:52 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: will@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 19:03:52 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202607 Will Andrews changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |will@FreeBSD.org --- Comment #20 from Will Andrews --- dbuf_rele_and_unlock() already contains logic that detects whether a dbuf is uncacheable and sets the db_is_ephemeral flag. Maybe it could be further extended to allow this flag to be set elsewhere, e.g. under the circumstances Justin proposes, and then for dbuf_rele_and_unlock() to perform the eviction at that point based on that state. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Wed Sep 23 19:18:01 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7D54A07AE8 for ; Wed, 23 Sep 2015 19:18:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C11211965 for ; Wed, 23 Sep 2015 19:18:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8NJI10V079841 for ; Wed, 23 Sep 2015 19:18:01 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 202607] [panic] Poudriere umounting file systems causes 'solaris assert: avl_is_empty(&dn->dn_dbufs)' panic Date: Wed, 23 Sep 2015 19:18:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: will@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 19:18:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202607 --- Comment #21 from Will Andrews --- Hmm, looks like I'm referring to local changes I forgot about. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Wed Sep 23 21:14:12 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 581D0A088A9 for ; Wed, 23 Sep 2015 21:14:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 45B3A1DC8 for ; Wed, 23 Sep 2015 21:14:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8NLECCG036447 for ; Wed, 23 Sep 2015 21:14:12 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 198457] zfs acl lost after zfs send-receive. Kernel panic Date: Wed, 23 Sep 2015 21:14:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: trasz@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 21:14:12 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198457 --- Comment #5 from Edward Tomasz Napierala --- (In reply to o.bende from comment #4) No idea. I'm guessing it's some kind of corruption, but one that for some reason only affects ACLs on send/receive. My advice about talking to Nexenta/ZoL folks is that because the code in question is not specific to FreeBSD, it might also affect them. They may have already found and fixed the bug. Otherwise... This is pure speculation, but user/group identifiers used natively by ZFS are somewhat complicated (ZFS supports various identifier schemes, eg Windows SIDs), and there might be some FUID-related problem? Again - shot in the dark. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Wed Sep 23 21:32:04 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C647A0712D for ; Wed, 23 Sep 2015 21:32:04 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-ig0-x236.google.com (mail-ig0-x236.google.com [IPv6:2607:f8b0:4001:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46D23199E for ; Wed, 23 Sep 2015 21:32:04 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by igbni9 with SMTP id ni9so36664228igb.0 for ; Wed, 23 Sep 2015 14:32:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LM1WQppPwAuBIGo508Rdd+xSV6pPb6cQmS6FxyMfxuc=; b=ZIWXlMvxZahgoSp2hwKt0Qn0wuuaPXwz1Z+O2X9ISltqA4/ARbJuNRpyNwuosb+h3e DS5D20aWlhik+8HE+6uLVCQBH/aiW4qcFUjgd0plnEeCGn+fnpwtXrsj4w2PvMFvdZD9 uL44pWl3C0q2kuFeHgRIAeKKgDMUIbRX7fVh+AH5g46aX4QzGOUh+Mq/j2SJiVnaInIF lBAsZK5TcM/Gtu9E+YsJ9vaNk9YY+slEWY2QxwFiTb5cIfhVFgGNNYd1XIXsM5HRTRMb TaJkjM3eLke92kCt9Jy4h+NxFpInCyZGAd6UEozPFqecR+Zst02rZ055PrShDsXxlAkn TBzA== MIME-Version: 1.0 X-Received: by 10.50.66.146 with SMTP id f18mr26146943igt.83.1443043923838; Wed, 23 Sep 2015 14:32:03 -0700 (PDT) Received: by 10.107.178.67 with HTTP; Wed, 23 Sep 2015 14:32:03 -0700 (PDT) In-Reply-To: References: <56023C15.7080007@fuckner.net> <5602D9E8.2090903@fuckner.net> Date: Wed, 23 Sep 2015 17:32:03 -0400 Message-ID: Subject: Re: Switching from MFI to MRSAS From: Ryan Stone To: Tim Gustafson Cc: Michael Fuckner , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 21:32:04 -0000 On Wed, Sep 23, 2015 at 12:59 PM, Tim Gustafson wrote: > The mrsas driver man page says that it reports the disks as "da0", > "da1" and so on, and the man pages about TRIM support all say that it > works with "ada" and "da" drives. So, I was hoping that the change in > driver technology would enable the TRIM command to work. > > I suppose I could update device.hints and loader.conf as describe, > reboot, and if the system doesn't come back up, just boot off a thumb > drive and re-edit those files back to their defaults... > The mrsas driver is interesting. Your RAID volumes show up as da0, da1, etc, but they really shouldn't. I'm doubtful that TRIM will work here because the drives are not truly directly attached, even though the mrsas driver is misleading the kernel into thinking that they are. You can always give a try, though. From owner-freebsd-fs@freebsd.org Wed Sep 23 22:27:04 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 91176A06DCF for ; Wed, 23 Sep 2015 22:27:04 +0000 (UTC) (envelope-from tjg@ucsc.edu) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 258F219C6 for ; Wed, 23 Sep 2015 22:27:04 +0000 (UTC) (envelope-from tjg@ucsc.edu) Received: by wicfx3 with SMTP id fx3so89249609wic.0 for ; Wed, 23 Sep 2015 15:27:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9E058vuH8N6Eswj3YDgHWpru2qm1TUpjw3Pa1QwVcl8=; b=VWRVvVmdcykZEKvxStQJ0JAdNPEbU8gyS7jfemCT5HGBpC2YbbOiz1C1TOBdzQUDBq nFWgs7Atg9TXAEdZHSfiC1zxYqdfSYRuGH3lEebS+6Ok4SKmGQSd5R56U4ZFAaZAi5N7 yQdhrvLK9YKZpQ3n2NtGlcKaLQLvIqFHJC25U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=9E058vuH8N6Eswj3YDgHWpru2qm1TUpjw3Pa1QwVcl8=; b=MBnVJiZLq1ZLg097+9ju9x/XXNfVk3Z32IQfExvTzEoLDR/CZlx2n5fkpCb5936olW n0cN3gXVwOnf+VDRwhx/P4rhtT41HNWslfSTvcUS2q0zYMoPnkBCWaWbtSVkiPrxocNb JCUR5diGvj+/MwANxGLfSMXNmLRTPRIOALlD8+IetyeOOi3J5/M1SEb6ufGetduLUnlI 76TpfnUoHMjorUtQd+LFhU4Nyk5u1Mm94pOfZOrF5wv2ro1ZCrkSPhz6dg3QfU2IcOyw phoeGNF59PORRw70EA6ToJ5xD2o39XTCbWiU45XVB2+TlnR8iwhLpXfHB6upmSk+zWZw Rp1A== X-Gm-Message-State: ALoCoQk/ChGVEoQq1JIcxvc/eK904BNyHk8SZ1GmtE7zugt298abvfys77SRSSQGso2goO6RY6v3 MIME-Version: 1.0 X-Received: by 10.180.8.106 with SMTP id q10mr5917425wia.92.1443047222693; Wed, 23 Sep 2015 15:27:02 -0700 (PDT) Received: by 10.194.92.209 with HTTP; Wed, 23 Sep 2015 15:27:02 -0700 (PDT) In-Reply-To: References: <56023C15.7080007@fuckner.net> <5602D9E8.2090903@fuckner.net> Date: Wed, 23 Sep 2015 15:27:02 -0700 Message-ID: Subject: Re: Switching from MFI to MRSAS From: Tim Gustafson To: Ryan Stone Cc: Michael Fuckner , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 22:27:04 -0000 > The mrsas driver is interesting. Your RAID volumes show up as da0, da1, > etc, but they really shouldn't. I'm doubtful that TRIM will work here > because the drives are not truly directly attached, even though the mrsas > driver is misleading the kernel into thinking that they are. > > You can always give a try, though. Just to be clear: we have the PERC controller configured in "pass-through" mode, which I understand to mean "HBA" mode. We are not using any of the RAID features of the PERC card. Instead, the server is configured as ZFS-on-root in a RAID 0+1 configuration: pool: tank state: ONLINE scan: scrub repaired 14.5K in 0h13m with 0 errors on Thu Sep 17 20:47:17 2015 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 mfisyspd0p3 ONLINE 0 0 0 mfisyspd1p3 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 mfisyspd3p3 ONLINE 0 0 0 mfisyspd2p3 ONLINE 0 0 0 where mfisyspd0, mfisyspd1, mfisyspd2 and mfisyspd3 are Samsung SSDs, not RAID volumes. So in this case, I believe it would be "correct" for the RAID controller to report the devices as "da0", "da1" and so on. -- Tim Gustafson tjg@ucsc.edu 831-459-5354 Baskin Engineering, Room 313A From owner-freebsd-fs@freebsd.org Wed Sep 23 22:37:23 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 94CE2A07362 for ; Wed, 23 Sep 2015 22:37:23 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wi0-f182.google.com (mail-wi0-f182.google.com [209.85.212.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32E2F1FC8 for ; Wed, 23 Sep 2015 22:37:22 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by wicfx3 with SMTP id fx3so89469699wic.0 for ; Wed, 23 Sep 2015 15:37:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=HbuFCmvkfNXMKuRC0/sN4+S2oWVBF9iPFsSEy4HI/QM=; b=j+/BIlh391h48+cq9VW5mskMs3F4QFMhB1GNL+sAVV+sMW55+16eCtJTqZI+D2p1Uj p+FrDWAce/mu5TjMI3GCY/wjBqsPVn5GYttos5EPVRuf0uervG2reRKINod3cS/0pbFt cNVo7B/vgWh9ndddbC+QtgEVAuMoobBHczsZ+4HOI39I5aLeXDMRDPsev0Y2fEi4XxgG 3XzJckzl9DjGAcDnOVRQmEUD2Od/kCByfdKRnFymzmzZ8JJcYALGsWzgfb39dUKSZWv+ 857gi46AUM2iYimRTaM9MupN6g5TJkK2VhBe+d7RbQ3BR57mwslX0c/NRP/Qtd3Nrk38 wcfQ== X-Gm-Message-State: ALoCoQkleLKpxBmahGFB9zHuD0GS71OjWwo9FLcPRZVlXinGmS/SEfpz+gwtxeCGIiF5sx69LiGM X-Received: by 10.194.93.198 with SMTP id cw6mr43418835wjb.113.1443047835336; Wed, 23 Sep 2015 15:37:15 -0700 (PDT) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by smtp.gmail.com with ESMTPSA id qc4sm9365944wjc.33.2015.09.23.15.37.14 for (version=TLSv1/SSLv3 cipher=OTHER); Wed, 23 Sep 2015 15:37:14 -0700 (PDT) Subject: Re: Switching from MFI to MRSAS To: freebsd-fs@freebsd.org References: <56023C15.7080007@fuckner.net> <5602D9E8.2090903@fuckner.net> From: Steven Hartland Message-ID: <56032993.50907@multiplay.co.uk> Date: Wed, 23 Sep 2015 23:37:07 +0100 User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 22:37:23 -0000 On 23/09/2015 22:32, Ryan Stone wrote: > On Wed, Sep 23, 2015 at 12:59 PM, Tim Gustafson wrote: > >> The mrsas driver man page says that it reports the disks as "da0", >> "da1" and so on, and the man pages about TRIM support all say that it >> works with "ada" and "da" drives. So, I was hoping that the change in >> driver technology would enable the TRIM command to work. >> >> I suppose I could update device.hints and loader.conf as describe, >> reboot, and if the system doesn't come back up, just boot off a thumb >> drive and re-edit those files back to their defaults... >> > The mrsas driver is interesting. Your RAID volumes show up as da0, da1, > etc, but they really shouldn't. I'm doubtful that TRIM will work here > because the drives are not truly directly attached, even though the mrsas > driver is misleading the kernel into thinking that they are. > > You can always give a try, though. camcontrol identify da0 will give you an indication ZFS will tell you straight away after a file delete: sysctl kstat.zfs.misc.zio_trim Regards Steve From owner-freebsd-fs@freebsd.org Thu Sep 24 07:30:20 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCC92A08172 for ; Thu, 24 Sep 2015 07:30:20 +0000 (UTC) (envelope-from hoerburger-richard@t-online.de) Received: from mailout06.t-online.de (mailout06.t-online.de [194.25.134.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7FB70196C for ; Thu, 24 Sep 2015 07:30:19 +0000 (UTC) (envelope-from hoerburger-richard@t-online.de) Received: from fwd35.aul.t-online.de (fwd35.aul.t-online.de [172.20.27.145]) by mailout06.t-online.de (Postfix) with SMTP id 8071931DAFA for ; Thu, 24 Sep 2015 09:30:11 +0200 (CEST) Received: from [192.168.1.253] (E2MnueZHrhTyzUmIta+4lPslPiR8lM4fryk600DxRrSrcVf34xuclJYjqjoJsRIQkP@[2.232.70.62]) by fwd35.t-online.de with (TLSv1:DHE-RSA-AES256-SHA encrypted) esmtp id 1Zf0yn-1cAmlU0; Thu, 24 Sep 2015 09:30:09 +0200 MIME-Version: 1.0 Subject: Warning!!! To: freebsd-fs@freebsd.org From: "E-Mail Administrator" Date: Thu, 24 Sep 2015 09:29:31 +0200 Message-ID: <1Zf0yn-1cAmlU0@fwd35.t-online.de> X-ID: E2MnueZHrhTyzUmIta+4lPslPiR8lM4fryk600DxRrSrcVf34xuclJYjqjoJsRIQkP X-TOI-MSGID: 7284a8b5-d594-45c4-996d-17eb86b1bf99 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 07:30:20 -0000 This notification was sent to freebsd-fs@freebsd.org Dear User: "freebsd-fs@freebsd.org", This is to notify you that you wont be able to send or receive mails . As you have used 98% of your quota and your account will be closed. The safety of your E-mail account is important. To avoid this from happening please click here to avoid suspension of E-= mail account. Warning!!! please do not ignore message to avoid your email -freebsd-fs@fr= eebsd.org- account from being closed. Thanks, Mail System Administrator. =20 From owner-freebsd-fs@freebsd.org Thu Sep 24 16:37:20 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 896A8A05968 for ; Thu, 24 Sep 2015 16:37:20 +0000 (UTC) (envelope-from pekka.jarvinen@gmail.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D451113C for ; Thu, 24 Sep 2015 16:37:20 +0000 (UTC) (envelope-from pekka.jarvinen@gmail.com) Received: by ioiz6 with SMTP id z6so82550922ioi.2 for ; Thu, 24 Sep 2015 09:37:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=JzTZMv1vH3++uIL7g10fpdSon3VuXkVK13i8b7Ka+vo=; b=h13VTnb2vJ0wcUjSJWxooFI/w0LzwhRHasvFCb19pQLIN6PpYHTjhTnpG6WlCCHsSC S9ENLZa/wQojHn5tt1KO1a+g78RBh+66mPVkGPtiVdHyNoJqw/L+43fgHIYqxwIsVRWv jXE87IIxy0ZqbrzUD8sRU8kjWppeY9hq3Sc0P62kxpUQE2AjUgk3YlF4+UmM4udiDlyL PVDzJDAGUowi19gN0dFqLeczpYL15Mit+kdTb/xVkE+/gjYfitNF95vZWyXrftcbChM9 5pt+1TTt6Jn+SPMs9NEIvVHN5JR+K3NkmTRigD5D60h2tKw03g8wzbPPdj8nH2jeYrt4 haHw== X-Received: by 10.107.31.135 with SMTP id f129mr1968896iof.8.1443112639659; Thu, 24 Sep 2015 09:37:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.92.6 with HTTP; Thu, 24 Sep 2015 09:36:50 -0700 (PDT) From: =?UTF-8?Q?Pekka_J=C3=A4rvinen?= Date: Thu, 24 Sep 2015 19:36:50 +0300 Message-ID: Subject: Missing free space from new raidz2 zpool To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 16:37:20 -0000 Hi, I got redirected here from serverfault. Original post: https://serverfault.com/questions/724542/missing-free-space-in-new-zfs-raid= z2-zpool So let's start from the beginning. I'm running Nas4Free 10.2.0.2 - Prester (revision 1814) which is based on FreeBSD 10.2-RELEASE-p2 #0 r287260M: Fri Aug 28 18:38:18 CEST 2015 System has 24 GB ECC memory. I bought 6 new 8 TB drives and created new raidz2 pool with these drives. Drives were only seen as 2.x TB. So I deleted the pool and moved the drives to plain SATA ports from SAS controller & SAS expander. Now drives are seen correctly by MB. My question is why zfs list and df is only showing 14.5T free? Shouldn't it be more close to 30T? Or is that old zfs metadata somehow lurking around and zfs is reading that? Bug? Thanks for help. ada0 at ata2 bus 0 scbus2 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada0: 7630885MB (15628053168 512 byte sectors: 16H 63S/T 16383C) ada1 at ata2 bus 0 scbus2 target 1 lun 0 ada1: ACS-2 ATA SATA 3.x device ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada1: 7630885MB (15628053168 512 byte sectors: 16H 63S/T 16383C) ada2 at ata3 bus 0 scbus3 target 0 lun 0 ada2: ACS-2 ATA SATA 3.x device ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada2: 7630885MB (15628053168 512 byte sectors: 16H 63S/T 16383C) ada3 at ata3 bus 0 scbus3 target 1 lun 0 ada3: ACS-2 ATA SATA 3.x device ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada3: 7630885MB (15628053168 512 byte sectors: 16H 63S/T 16383C) ada4 at ata4 bus 0 scbus4 target 0 lun 0 ada4: ACS-2 ATA SATA 3.x device ada4: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada4: 7630885MB (15628053168 512 byte sectors: 16H 63S/T 16383C) ada5 at ata5 bus 0 scbus5 target 0 lun 0 ada5: ACS-2 ATA SATA 3.x device ada5: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada5: 7630885MB (15628053168 512 byte sectors: 16H 63S/T 16383C) # zpool create storage2 raidz2 ada0 ada1 ada2 ada3 ada4 ada5 # zpool status pool: storage2 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM storage2 ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 ada0 ONLINE 0 0 0 ada1 ONLINE 0 0 0 ada2 ONLINE 0 0 0 ada3 ONLINE 0 0 0 ada4 ONLINE 0 0 0 ada5 ONLINE 0 0 0 errors: No known data errors # zpool iostat storage2 capacity operations bandwidth pool alloc free read write read write ---------- ----- ----- ----- ----- ----- ----- storage2 900K 43.5T 0 1 0 7.84K # zfs list -t all NAME USED AVAIL REFER MOUNTPOINT storage2 480K 14.5T 192K /storage2 # zfs get all storage2 NAME PROPERTY VALUE SOURCE storage2 type filesystem - storage2 creation Thu Sep 24 18:48 2015 - storage2 used 623K - storage2 available 14.5T - storage2 referenced 192K - storage2 compressratio 1.00x - storage2 mounted yes - storage2 quota none default storage2 reservation none default storage2 recordsize 128K default storage2 mountpoint /storage2 default storage2 sharenfs off default storage2 checksum on default storage2 compression off default storage2 atime on default storage2 devices on default storage2 exec on default storage2 setuid on default storage2 readonly off default storage2 jailed off default storage2 snapdir hidden default storage2 aclmode discard default storage2 aclinherit restricted default storage2 canmount on default storage2 xattr off temporary storage2 copies 1 default storage2 version 5 - storage2 utf8only off - storage2 normalization none - storage2 casesensitivity sensitive - storage2 vscan off default storage2 nbmand off default storage2 sharesmb off default storage2 refquota none default storage2 refreservation none default storage2 primarycache all default storage2 secondarycache all default storage2 usedbysnapshots 0 - storage2 usedbydataset 192K - storage2 usedbychildren 432K - storage2 usedbyrefreservation 0 - storage2 logbias latency default storage2 dedup off default storage2 mlslabel - storage2 sync standard default storage2 refcompressratio 1.00x - storage2 written 192K - storage2 logicalused 80K - storage2 logicalreferenced 9.50K - storage2 volmode default default storage2 filesystem_limit none default storage2 snapshot_limit none default storage2 filesystem_count none default storage2 snapshot_count none default storage2 redundant_metadata all default # zdb storage2 Cached configuration: version: 5000 name: 'storage2' state: 0 txg: 4 pool_guid: 10414888771172952257 hostid: 1452759488 hostname: 'storage.home' vdev_children: 1 vdev_tree: type: 'root' id: 0 guid: 10414888771172952257 create_txg: 4 children[0]: type: 'raidz' id: 0 guid: 2681305684837460215 nparity: 2 metaslab_array: 34 metaslab_shift: 38 ashift: 12 asize: 48009350479872 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 14137175296949034209 path: '/dev/ada0' phys_path: '/dev/ada0' whole_disk: 1 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 5793672826394463181 path: '/dev/ada1' phys_path: '/dev/ada1' whole_disk: 1 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 426194980956466479 path: '/dev/ada2' phys_path: '/dev/ada2' whole_disk: 1 create_txg: 4 children[3]: type: 'disk' id: 3 guid: 5113566757325497587 path: '/dev/ada3' phys_path: '/dev/ada3' whole_disk: 1 create_txg: 4 children[4]: type: 'disk' id: 4 guid: 16297810626525192704 path: '/dev/ada4' phys_path: '/dev/ada4' whole_disk: 1 create_txg: 4 children[5]: type: 'disk' id: 5 guid: 16307112257360078181 path: '/dev/ada5' phys_path: '/dev/ada5' whole_disk: 1 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data MOS Configuration: version: 5000 name: 'storage2' state: 0 txg: 4 pool_guid: 10414888771172952257 hostid: 1452759488 hostname: 'storage.home' vdev_children: 1 vdev_tree: type: 'root' id: 0 guid: 10414888771172952257 create_txg: 4 children[0]: type: 'raidz' id: 0 guid: 2681305684837460215 nparity: 2 metaslab_array: 34 metaslab_shift: 38 ashift: 12 asize: 48009350479872 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 14137175296949034209 path: '/dev/ada0' phys_path: '/dev/ada0' whole_disk: 1 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 5793672826394463181 path: '/dev/ada1' phys_path: '/dev/ada1' whole_disk: 1 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 426194980956466479 path: '/dev/ada2' phys_path: '/dev/ada2' whole_disk: 1 create_txg: 4 children[3]: type: 'disk' id: 3 guid: 5113566757325497587 path: '/dev/ada3' phys_path: '/dev/ada3' whole_disk: 1 create_txg: 4 children[4]: type: 'disk' id: 4 guid: 16297810626525192704 path: '/dev/ada4' phys_path: '/dev/ada4' whole_disk: 1 create_txg: 4 children[5]: type: 'disk' id: 5 guid: 16307112257360078181 path: '/dev/ada5' phys_path: '/dev/ada5' whole_disk: 1 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data Uberblock: magic =3D 0000000000bab10c version =3D 5000 txg =3D 5 guid_sum =3D 15831494980392489965 timestamp =3D 1443109719 UTC =3D Thu Sep 24 18:48:39 2015 All DDTs are empty Metaslabs: vdev 0 metaslabs 174 offset spacemap free --------------- ------------------- --------------- ------------- metaslab 0 offset 0 spacemap 37 free 256G On-disk histogram: fragmentation 0 37: 1 * metaslab 1 offset 4000000000 spacemap 0 free 256G metaslab 2 offset 8000000000 spacemap 0 free 256G metaslab 3 offset c000000000 spacemap 0 free 256G metaslab 4 offset 10000000000 spacemap 0 free 256G metaslab 5 offset 14000000000 spacemap 0 free 256G metaslab 6 offset 18000000000 spacemap 0 free 256G metaslab 7 offset 1c000000000 spacemap 0 free 256G metaslab 8 offset 20000000000 spacemap 0 free 256G metaslab 9 offset 24000000000 spacemap 0 free 256G metaslab 10 offset 28000000000 spacemap 0 free 256G metaslab 11 offset 2c000000000 spacemap 0 free 256G metaslab 12 offset 30000000000 spacemap 0 free 256G metaslab 13 offset 34000000000 spacemap 0 free 256G metaslab 14 offset 38000000000 spacemap 0 free 256G metaslab 15 offset 3c000000000 spacemap 0 free 256G metaslab 16 offset 40000000000 spacemap 0 free 256G metaslab 17 offset 44000000000 spacemap 0 free 256G metaslab 18 offset 48000000000 spacemap 0 free 256G metaslab 19 offset 4c000000000 spacemap 0 free 256G metaslab 20 offset 50000000000 spacemap 0 free 256G metaslab 21 offset 54000000000 spacemap 0 free 256G metaslab 22 offset 58000000000 spacemap 0 free 256G metaslab 23 offset 5c000000000 spacemap 0 free 256G metaslab 24 offset 60000000000 spacemap 0 free 256G metaslab 25 offset 64000000000 spacemap 0 free 256G metaslab 26 offset 68000000000 spacemap 0 free 256G metaslab 27 offset 6c000000000 spacemap 0 free 256G metaslab 28 offset 70000000000 spacemap 0 free 256G metaslab 29 offset 74000000000 spacemap 0 free 256G metaslab 30 offset 78000000000 spacemap 0 free 256G metaslab 31 offset 7c000000000 spacemap 0 free 256G metaslab 32 offset 80000000000 spacemap 0 free 256G metaslab 33 offset 84000000000 spacemap 36 free 256G On-disk histogram: fragmentation 0 37: 1 * metaslab 34 offset 88000000000 spacemap 0 free 256G metaslab 35 offset 8c000000000 spacemap 0 free 256G metaslab 36 offset 90000000000 spacemap 0 free 256G metaslab 37 offset 94000000000 spacemap 0 free 256G metaslab 38 offset 98000000000 spacemap 0 free 256G metaslab 39 offset 9c000000000 spacemap 0 free 256G metaslab 40 offset a0000000000 spacemap 0 free 256G metaslab 41 offset a4000000000 spacemap 0 free 256G metaslab 42 offset a8000000000 spacemap 0 free 256G metaslab 43 offset ac000000000 spacemap 0 free 256G metaslab 44 offset b0000000000 spacemap 0 free 256G metaslab 45 offset b4000000000 spacemap 0 free 256G metaslab 46 offset b8000000000 spacemap 0 free 256G metaslab 47 offset bc000000000 spacemap 0 free 256G metaslab 48 offset c0000000000 spacemap 0 free 256G metaslab 49 offset c4000000000 spacemap 0 free 256G metaslab 50 offset c8000000000 spacemap 0 free 256G metaslab 51 offset cc000000000 spacemap 0 free 256G metaslab 52 offset d0000000000 spacemap 0 free 256G metaslab 53 offset d4000000000 spacemap 0 free 256G metaslab 54 offset d8000000000 spacemap 0 free 256G metaslab 55 offset dc000000000 spacemap 0 free 256G metaslab 56 offset e0000000000 spacemap 0 free 256G metaslab 57 offset e4000000000 spacemap 0 free 256G metaslab 58 offset e8000000000 spacemap 0 free 256G metaslab 59 offset ec000000000 spacemap 0 free 256G metaslab 60 offset f0000000000 spacemap 0 free 256G metaslab 61 offset f4000000000 spacemap 0 free 256G metaslab 62 offset f8000000000 spacemap 0 free 256G metaslab 63 offset fc000000000 spacemap 0 free 256G metaslab 64 offset 100000000000 spacemap 0 free 256G metaslab 65 offset 104000000000 spacemap 0 free 256G metaslab 66 offset 108000000000 spacemap 35 free 256G On-disk histogram: fragmentation 0 37: 1 * metaslab 67 offset 10c000000000 spacemap 0 free 256G metaslab 68 offset 110000000000 spacemap 0 free 256G metaslab 69 offset 114000000000 spacemap 0 free 256G metaslab 70 offset 118000000000 spacemap 0 free 256G metaslab 71 offset 11c000000000 spacemap 0 free 256G metaslab 72 offset 120000000000 spacemap 0 free 256G metaslab 73 offset 124000000000 spacemap 0 free 256G metaslab 74 offset 128000000000 spacemap 0 free 256G metaslab 75 offset 12c000000000 spacemap 0 free 256G metaslab 76 offset 130000000000 spacemap 0 free 256G metaslab 77 offset 134000000000 spacemap 0 free 256G metaslab 78 offset 138000000000 spacemap 0 free 256G metaslab 79 offset 13c000000000 spacemap 0 free 256G metaslab 80 offset 140000000000 spacemap 0 free 256G metaslab 81 offset 144000000000 spacemap 0 free 256G metaslab 82 offset 148000000000 spacemap 0 free 256G metaslab 83 offset 14c000000000 spacemap 0 free 256G metaslab 84 offset 150000000000 spacemap 0 free 256G metaslab 85 offset 154000000000 spacemap 0 free 256G metaslab 86 offset 158000000000 spacemap 0 free 256G metaslab 87 offset 15c000000000 spacemap 0 free 256G metaslab 88 offset 160000000000 spacemap 0 free 256G metaslab 89 offset 164000000000 spacemap 0 free 256G metaslab 90 offset 168000000000 spacemap 0 free 256G metaslab 91 offset 16c000000000 spacemap 0 free 256G metaslab 92 offset 170000000000 spacemap 0 free 256G metaslab 93 offset 174000000000 spacemap 0 free 256G metaslab 94 offset 178000000000 spacemap 0 free 256G metaslab 95 offset 17c000000000 spacemap 0 free 256G metaslab 96 offset 180000000000 spacemap 0 free 256G metaslab 97 offset 184000000000 spacemap 0 free 256G metaslab 98 offset 188000000000 spacemap 0 free 256G metaslab 99 offset 18c000000000 spacemap 0 free 256G metaslab 100 offset 190000000000 spacemap 0 free 256G metaslab 101 offset 194000000000 spacemap 0 free 256G metaslab 102 offset 198000000000 spacemap 0 free 256G metaslab 103 offset 19c000000000 spacemap 0 free 256G metaslab 104 offset 1a0000000000 spacemap 0 free 256G metaslab 105 offset 1a4000000000 spacemap 0 free 256G metaslab 106 offset 1a8000000000 spacemap 0 free 256G metaslab 107 offset 1ac000000000 spacemap 0 free 256G metaslab 108 offset 1b0000000000 spacemap 0 free 256G metaslab 109 offset 1b4000000000 spacemap 0 free 256G metaslab 110 offset 1b8000000000 spacemap 0 free 256G metaslab 111 offset 1bc000000000 spacemap 0 free 256G metaslab 112 offset 1c0000000000 spacemap 0 free 256G metaslab 113 offset 1c4000000000 spacemap 0 free 256G metaslab 114 offset 1c8000000000 spacemap 0 free 256G metaslab 115 offset 1cc000000000 spacemap 0 free 256G metaslab 116 offset 1d0000000000 spacemap 0 free 256G metaslab 117 offset 1d4000000000 spacemap 0 free 256G metaslab 118 offset 1d8000000000 spacemap 0 free 256G metaslab 119 offset 1dc000000000 spacemap 0 free 256G metaslab 120 offset 1e0000000000 spacemap 0 free 256G metaslab 121 offset 1e4000000000 spacemap 0 free 256G metaslab 122 offset 1e8000000000 spacemap 0 free 256G metaslab 123 offset 1ec000000000 spacemap 0 free 256G metaslab 124 offset 1f0000000000 spacemap 0 free 256G metaslab 125 offset 1f4000000000 spacemap 0 free 256G metaslab 126 offset 1f8000000000 spacemap 0 free 256G metaslab 127 offset 1fc000000000 spacemap 0 free 256G metaslab 128 offset 200000000000 spacemap 0 free 256G metaslab 129 offset 204000000000 spacemap 0 free 256G metaslab 130 offset 208000000000 spacemap 0 free 256G metaslab 131 offset 20c000000000 spacemap 0 free 256G metaslab 132 offset 210000000000 spacemap 0 free 256G metaslab 133 offset 214000000000 spacemap 0 free 256G metaslab 134 offset 218000000000 spacemap 0 free 256G metaslab 135 offset 21c000000000 spacemap 0 free 256G metaslab 136 offset 220000000000 spacemap 0 free 256G metaslab 137 offset 224000000000 spacemap 0 free 256G metaslab 138 offset 228000000000 spacemap 0 free 256G metaslab 139 offset 22c000000000 spacemap 0 free 256G metaslab 140 offset 230000000000 spacemap 0 free 256G metaslab 141 offset 234000000000 spacemap 0 free 256G metaslab 142 offset 238000000000 spacemap 0 free 256G metaslab 143 offset 23c000000000 spacemap 0 free 256G metaslab 144 offset 240000000000 spacemap 0 free 256G metaslab 145 offset 244000000000 spacemap 0 free 256G metaslab 146 offset 248000000000 spacemap 0 free 256G metaslab 147 offset 24c000000000 spacemap 0 free 256G metaslab 148 offset 250000000000 spacemap 0 free 256G metaslab 149 offset 254000000000 spacemap 0 free 256G metaslab 150 offset 258000000000 spacemap 0 free 256G metaslab 151 offset 25c000000000 spacemap 0 free 256G metaslab 152 offset 260000000000 spacemap 0 free 256G metaslab 153 offset 264000000000 spacemap 0 free 256G metaslab 154 offset 268000000000 spacemap 0 free 256G metaslab 155 offset 26c000000000 spacemap 0 free 256G metaslab 156 offset 270000000000 spacemap 0 free 256G metaslab 157 offset 274000000000 spacemap 0 free 256G metaslab 158 offset 278000000000 spacemap 0 free 256G metaslab 159 offset 27c000000000 spacemap 0 free 256G metaslab 160 offset 280000000000 spacemap 0 free 256G metaslab 161 offset 284000000000 spacemap 0 free 256G metaslab 162 offset 288000000000 spacemap 0 free 256G metaslab 163 offset 28c000000000 spacemap 0 free 256G metaslab 164 offset 290000000000 spacemap 0 free 256G metaslab 165 offset 294000000000 spacemap 0 free 256G metaslab 166 offset 298000000000 spacemap 0 free 256G metaslab 167 offset 29c000000000 spacemap 0 free 256G metaslab 168 offset 2a0000000000 spacemap 0 free 256G metaslab 169 offset 2a4000000000 spacemap 0 free 256G metaslab 170 offset 2a8000000000 spacemap 0 free 256G metaslab 171 offset 2ac000000000 spacemap 0 free 256G metaslab 172 offset 2b0000000000 spacemap 0 free 256G metaslab 173 offset 2b4000000000 spacemap 0 free 256G vdev 0 metaslabs 174 fragmentation 0% 37: 3 *** pool storage2 fragmentation 0% 37: 3 *** Dataset mos [META], ID 0, cr_txg 4, 432K, 37 objects Object lvl iblk dblk dsize lsize %full type 0 1 16K 16K 96.0K 32K 57.81 DMU dnode 1 1 16K 1K 24.0K 1K 100.00 object directory 2 1 16K 512 0 512 0.00 DSL directory 3 1 16K 512 0 512 100.00 DSL props 4 1 16K 512 0 512 100.00 DSL directory child ma= p 5 1 16K 512 0 512 0.00 DSL directory 6 1 16K 512 0 512 100.00 DSL props 7 1 16K 512 0 512 100.00 DSL directory child ma= p 8 1 16K 512 0 512 0.00 DSL directory 9 1 16K 512 0 512 100.00 DSL props 10 1 16K 512 0 512 100.00 DSL directory child ma= p 11 1 16K 128K 0 128K 0.00 bpobj 12 1 16K 512 0 512 0.00 DSL directory 13 1 16K 512 0 512 100.00 DSL props 14 1 16K 512 0 512 100.00 DSL directory child ma= p 15 1 16K 512 0 512 0.00 DSL dataset 16 1 16K 512 0 512 100.00 DSL dataset snap map 17 1 16K 512 0 512 100.00 DSL deadlist map 18 1 16K 512 0 512 0.00 DSL dataset 19 1 16K 512 0 512 100.00 DSL deadlist map 20 1 16K 128K 0 128K 0.00 bpobj 21 1 16K 512 0 512 0.00 DSL dataset 22 1 16K 512 0 512 100.00 DSL dataset snap map 23 1 16K 512 0 512 100.00 DSL deadlist map 24 1 16K 128K 0 128K 0.00 bpobj 25 1 16K 512 0 512 100.00 DSL dataset next clone= s 26 1 16K 512 0 512 100.00 DSL dir clones 27 1 16K 16K 24.0K 16K 100.00 packed nvlist 28 1 16K 512 24.0K 512 100.00 zap 29 1 16K 512 24.0K 512 100.00 zap 30 1 16K 16K 48.0K 32K 100.00 zap 31 1 16K 16K 48.0K 16K 100.00 bpobj (Z=3Duncompresse= d) 32 1 16K 128K 24.0K 128K 100.00 SPA history 33 1 16K 512 24.0K 512 100.00 zap 34 1 16K 512 0 1K 100.00 object array 35 1 16K 4K 24.0K 4K 100.00 SPA space map 36 1 16K 4K 24.0K 4K 100.00 SPA space map 37 1 16K 4K 24.0K 4K 100.00 SPA space map Dataset storage2 [ZPL], ID 21, cr_txg 1, 192K, 7 objects Object lvl iblk dblk dsize lsize %full type 0 7 16K 16K 112K 16K 21.88 DMU dnode -1 1 16K 512 0 512 100.00 ZFS user/group used -2 1 16K 512 0 512 100.00 ZFS user/group used 1 1 16K 1K 16K 1K 100.00 ZFS master node 2 1 16K 512 0 512 100.00 SA master node 3 1 16K 512 0 512 100.00 ZFS delete queue 4 1 16K 512 0 512 100.00 ZFS directory 5 1 16K 1.50K 16K 1.50K 100.00 SA attr registration 6 1 16K 16K 32K 32K 100.00 SA attr layouts 7 1 16K 512 0 512 100.00 ZFS directory Verified large_blocks feature refcount is correct (0) Traversing all blocks to verify checksums and verify nothing leaked ... loading space map for vdev 0 of 1, metaslab 66 of 174 ... No leaks (block sum matches space maps exactly) bp count: 60 ganged count: 0 bp logical: 545280 avg: 9088 bp physical: 92160 avg: 1536 compression: 5.92 bp allocated: 1327104 avg: 22118 compression: 0.41 bp deduped: 0 ref>1: 0 deduplication: 1.00 SPA allocated: 1327104 used: 0.00% additional, non-pointer bps of type 0: 23 Dittoed blocks on same vdev: 37 Blocks LSIZE PSIZE ASIZE avg comp %Total Type - - - - - - - unallocated 1 1K 512 36.0K 36.0K 2.00 2.78 object directory - - - - - - - object array 1 16K 1K 36.0K 36.0K 16.00 2.78 packed nvlist - - - - - - - packed nvlist size 1 16K 16K 72.0K 72.0K 1.00 5.56 bpobj - - - - - - - bpobj header - - - - - - - SPA space map heade= r 3 12.0K 12.0K 108K 36.0K 1.00 8.33 SPA space map - - - - - - - ZIL intent log 9 144K 36.0K 312K 34.7K 4.00 24.07 DMU dnode 2 4K 2.50K 60.0K 30.0K 1.60 4.63 DMU objset - - - - - - - DSL directory - - - - - - - DSL directory child map - - - - - - - DSL dataset snap ma= p - - - - - - - DSL props - - - - - - - DSL dataset - - - - - - - ZFS znode - - - - - - - ZFS V0 ACL - - - - - - - ZFS plain file - - - - - - - ZFS directory 1 1K 512 24.0K 24.0K 2.00 1.85 ZFS master node - - - - - - - ZFS delete queue - - - - - - - zvol object - - - - - - - zvol prop - - - - - - - other uint8[] - - - - - - - other uint64[] - - - - - - - other ZAP - - - - - - - persistent error lo= g 1 128K 1K 36.0K 36.0K 128.00 2.78 SPA history - - - - - - - SPA history offsets - - - - - - - Pool properties - - - - - - - DSL permissions - - - - - - - ZFS ACL - - - - - - - ZFS SYSACL - - - - - - - FUID table - - - - - - - FUID table size - - - - - - - DSL dataset next clones - - - - - - - scan work queue - - - - - - - ZFS user/group used - - - - - - - ZFS user/group quot= a - - - - - - - snapshot refcount tags - - - - - - - DDT ZAP algorithm - - - - - - - DDT statistics - - - - - - - System attributes - - - - - - - SA master node 1 1.50K 512 24.0K 24.0K 3.00 1.85 SA attr registratio= n 2 32K 4K 48.0K 24.0K 8.00 3.70 SA attr layouts - - - - - - - scan translations - - - - - - - deduplicated block - - - - - - - DSL deadlist map - - - - - - - DSL deadlist map hd= r - - - - - - - DSL dir clones - - - - - - - bpobj subobj 10 132K 10.0K 360K 36.0K 13.20 27.78 deferred free - - - - - - - dedup ditto 5 33.5K 6.00K 180K 36.0K 5.58 13.89 other 60 533K 90.0K 1.27M 21.6K 5.92 100.00 Total capacity operations bandwidth ---- errors ---- description used avail read write read write read write cksum storage2 1.27M 43.5T 169 0 928K 0 0 0 0 raidz2 1.27M 43.5T 169 0 928K 0 0 0 0 /dev/ada0 100 0 1.25M 0 0 0 0 /dev/ada1 96 0 828K 0 0 0 0 /dev/ada2 111 0 888K 0 0 0 0 /dev/ada3 93 0 816K 0 0 0 0 /dev/ada4 93 0 816K 0 0 0 0 /dev/ada5 109 0 880K 0 0 0 0 History: unrecognized record: history internal str: 'pool version 5000; software version 5000/5; uts storage.home 10.2-RELEASE-p2 1002000 amd64' internal_name: 'create' history txg: 5 history time: 1443109719 history hostname: 'storage.home' 2015-09-24.18:48:39 zpool create storage2 raidz2 ada0 ada1 ada2 ada3 ada4 ada5 history command: 'zpool create storage2 raidz2 ada0 ada1 ada2 ada3 ada4 ada5' history who: 0 history time: 1443109719 history hostname: 'storage.home' --=20 Pekka J=C3=A4rvinen From owner-freebsd-fs@freebsd.org Fri Sep 25 09:08:37 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 574FFA09E4B for ; Fri, 25 Sep 2015 09:08:37 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E1E711AA0 for ; Fri, 25 Sep 2015 09:08:36 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id EF6CF153430 for ; Fri, 25 Sep 2015 11:08:31 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4DH3eC6cM9cQ; Fri, 25 Sep 2015 11:08:31 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:945d:46a:6715:8250] (unknown [IPv6:2001:4cb8:3:1:945d:46a:6715:8250]) by smtp.digiware.nl (Postfix) with ESMTP id 3115F153418 for ; Fri, 25 Sep 2015 11:08:31 +0200 (CEST) To: "freebsd-fs@freebsd.org" From: Willem Jan Withagen Subject: ZFS, Zvol, iSCSI and windows Organization: Digiware Management b.v. Message-ID: <56050EFC.1000303@digiware.nl> Date: Fri, 25 Sep 2015 11:08:12 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Sep 2015 09:08:37 -0000 Hi, Because of the Network Video Recorder (on windows) we use only likes "real" disks, and not SMB disks. We started using ZVOLs which are exported thru iSCSI/ctld.... And that works really well, so there all thumbs up for this combo. However..... (you knew that was coming) I do have some questions, and hope somebody can share some insights. 1) This is a ZFS question. I have created a ZVOL with: zfs create -s -V 5T zfsraid/nvr2 Looking at the disk usage in 'zfs get all': NAME PROPERTY VALUE SOURCE zfsraid/nvr2 used 9.97T - zfsraid/nvr2 available 438G - zfsraid/nvr2 referenced 9.97T - zfsraid/nvr2 compressratio 1.00x - zfsraid/nvr2 reservation none default zfsraid/nvr2 volsize 5T local zfsraid/nvr2 volblocksize 8K - zfsraid/nvr2 checksum on default zfsraid/nvr2 compression lz4 default zfsraid/nvr2 primarycache all default zfsraid/nvr2 secondarycache all default zfsraid/nvr2 usedbysnapshots 0 - zfsraid/nvr2 usedbydataset 9.97T - zfsraid/nvr2 usedbychildren 0 - zfsraid/nvr2 usedbyrefreservation 0 - zfsraid/nvr2 sync standard default zfsraid/nvr2 written 9.97T - zfsraid/nvr2 logicalused 4.97T - zfsraid/nvr2 logicalreferenced 4.97T - zfsraid/nvr2 volmode default default And what sort of "worries" me is that it seems that this volume is using twice the amount of diskspace it is offering as ZVOL? a) Is this really true? b) Should I have done something different to not waste so much overhead? Note that the compression rate is 1.00x, which is of course due to writing h264 media streams that do not compress at all. But is the parents default, and I haven't turned it off. 2) The second one might be more a Windows question, but anyways. The export is that same 5T ZVOL, plain create with zfs create -s -V 5T zfsraid/nvr2 Under Windows I used the regular stuff to format the dis with GPT and NTFS with 8k segments. (matching the ZVOL blocksize) Upon reboot FreeBSD notices the following: GEOM_PART: partition 1 on (zvol/zfsraid/nvr2, GPT) is not aligned on 8192 bytes By itself not technical problem. But the warning does hint that the alignment is thus, that performance might suffer. the backing disks are WD REDs which are 4K sectors.... So the misalignment could cause too many read/writes for writing. Does anybody have a clue as how to get Windows to do the aligning correctly? From owner-freebsd-fs@freebsd.org Fri Sep 25 11:13:45 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 170E5A095C4 for ; Fri, 25 Sep 2015 11:13:45 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from Exch2-3.slu.se (exch2-3.slu.se [77.235.224.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A243F1AF9 for ; Fri, 25 Sep 2015 11:13:44 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by Exch2-3.slu.se (77.235.224.123) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Fri, 25 Sep 2015 13:13:34 +0200 Received: from exch2-4.slu.se ([fe80::bc24:95e8:1e1e:f3ae]) by exch2-4.slu.se ([fe80::bc24:95e8:1e1e:f3ae%22]) with mapi id 15.00.1104.000; Fri, 25 Sep 2015 13:13:34 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Willem Jan Withagen CC: "freebsd-fs@freebsd.org" Subject: Re: ZFS, Zvol, iSCSI and windows Thread-Topic: ZFS, Zvol, iSCSI and windows Thread-Index: AQHQ93HLAK1gWL7n30CZVpf4MzfBnJ5M9gQA Date: Fri, 25 Sep 2015 11:13:34 +0000 Message-ID: <1443179614.5271.42.camel@data-b104.adm.slu.se> References: <56050EFC.1000303@digiware.nl> In-Reply-To: <56050EFC.1000303@digiware.nl> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [77.235.228.63] Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Sep 2015 11:13:45 -0000 ZnJlIDIwMTUtMDktMjUga2xvY2thbiAxMTowOCArMDIwMCBza3JldiBXaWxsZW0gSmFuIFdpdGhh Z2VuOg0KPiBIaSwNCj4gDQo+IEJlY2F1c2Ugb2YgdGhlIE5ldHdvcmsgVmlkZW8gUmVjb3JkZXIg KG9uIHdpbmRvd3MpIHdlIHVzZSBvbmx5IGxpa2VzIA0KPiAicmVhbCIgZGlza3MsIGFuZCBub3Qg U01CIGRpc2tzLiBXZSBzdGFydGVkIHVzaW5nIFpWT0xzIHdoaWNoIGFyZSANCj4gZXhwb3J0ZWQg dGhydSBpU0NTSS9jdGxkLi4uLg0KPiANCj4gQW5kIHRoYXQgd29ya3MgcmVhbGx5IHdlbGwsIHNv IHRoZXJlIGFsbCB0aHVtYnMgdXAgZm9yIHRoaXMgY29tYm8uDQo+IA0KPiBIb3dldmVyLi4uLi4g KHlvdSBrbmV3IHRoYXQgd2FzIGNvbWluZykNCj4gSSBkbyBoYXZlIHNvbWUgcXVlc3Rpb25zLCBh bmQgaG9wZSBzb21lYm9keSBjYW4gc2hhcmUgc29tZSBpbnNpZ2h0cy4NCj4gDQo+IDEpDQo+IFRo aXMgaXMgYSBaRlMgcXVlc3Rpb24uDQo+IA0KPiBJIGhhdmUgY3JlYXRlZCBhIFpWT0wgd2l0aDoN Cj4gCXpmcyBjcmVhdGUgLXMgLVYgNVQgemZzcmFpZC9udnIyDQo+IA0KPiBMb29raW5nIGF0IHRo ZSBkaXNrIHVzYWdlIGluICd6ZnMgZ2V0IGFsbCc6DQo+IE5BTUUgICAgICAgICAgUFJPUEVSVFkg ICAgICAgICAgICAgIFZBTFVFICAgICAgICAgICAgICAgICAgU09VUkNFDQo+IHpmc3JhaWQvbnZy MiAgdXNlZCAgICAgICAgICAgICAgICAgIDkuOTdUICAgICAgICAgICAgICAgICAgLQ0KPiB6ZnNy YWlkL252cjIgIGF2YWlsYWJsZSAgICAgICAgICAgICA0MzhHICAgICAgICAgICAgICAgICAgIC0N Cj4gemZzcmFpZC9udnIyICByZWZlcmVuY2VkICAgICAgICAgICAgOS45N1QgICAgICAgICAgICAg ICAgICAtDQo+IHpmc3JhaWQvbnZyMiAgY29tcHJlc3NyYXRpbyAgICAgICAgIDEuMDB4ICAgICAg ICAgICAgICAgICAgLQ0KPiB6ZnNyYWlkL252cjIgIHJlc2VydmF0aW9uICAgICAgICAgICBub25l ICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCj4gemZzcmFpZC9udnIyICB2b2xzaXplICAgICAg ICAgICAgICAgNVQgICAgICAgICAgICAgICAgICAgICBsb2NhbA0KPiB6ZnNyYWlkL252cjIgIHZv bGJsb2Nrc2l6ZSAgICAgICAgICA4SyAgICAgICAgICAgICAgICAgICAgIC0NCj4gemZzcmFpZC9u dnIyICBjaGVja3N1bSAgICAgICAgICAgICAgb24gICAgICAgICAgICAgICAgICAgICBkZWZhdWx0 DQo+IHpmc3JhaWQvbnZyMiAgY29tcHJlc3Npb24gICAgICAgICAgIGx6NCAgICAgICAgICAgICAg ICAgICAgZGVmYXVsdA0KPiB6ZnNyYWlkL252cjIgIHByaW1hcnljYWNoZSAgICAgICAgICBhbGwg ICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCj4gemZzcmFpZC9udnIyICBzZWNvbmRhcnljYWNo ZSAgICAgICAgYWxsICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQo+IHpmc3JhaWQvbnZyMiAg dXNlZGJ5c25hcHNob3RzICAgICAgIDAgICAgICAgICAgICAgICAgICAgICAgLQ0KPiB6ZnNyYWlk L252cjIgIHVzZWRieWRhdGFzZXQgICAgICAgICA5Ljk3VCAgICAgICAgICAgICAgICAgIC0NCj4g emZzcmFpZC9udnIyICB1c2VkYnljaGlsZHJlbiAgICAgICAgMCAgICAgICAgICAgICAgICAgICAg ICAtDQo+IHpmc3JhaWQvbnZyMiAgdXNlZGJ5cmVmcmVzZXJ2YXRpb24gIDAgICAgICAgICAgICAg ICAgICAgICAgLQ0KPiB6ZnNyYWlkL252cjIgIHN5bmMgICAgICAgICAgICAgICAgICBzdGFuZGFy ZCAgICAgICAgICAgICAgIGRlZmF1bHQNCj4gemZzcmFpZC9udnIyICB3cml0dGVuICAgICAgICAg ICAgICAgOS45N1QgICAgICAgICAgICAgICAgICAtDQo+IHpmc3JhaWQvbnZyMiAgbG9naWNhbHVz ZWQgICAgICAgICAgIDQuOTdUICAgICAgICAgICAgICAgICAgLQ0KPiB6ZnNyYWlkL252cjIgIGxv Z2ljYWxyZWZlcmVuY2VkICAgICA0Ljk3VCAgICAgICAgICAgICAgICAgIC0NCj4gemZzcmFpZC9u dnIyICB2b2xtb2RlICAgICAgICAgICAgICAgZGVmYXVsdCAgICAgICAgICAgICAgICBkZWZhdWx0 DQo+IA0KPiBBbmQgd2hhdCBzb3J0IG9mICJ3b3JyaWVzIiBtZSBpcyB0aGF0IGl0IHNlZW1zIHRo YXQgdGhpcyB2b2x1bWUgaXMgdXNpbmcgDQo+IHR3aWNlIHRoZSBhbW91bnQgb2YgZGlza3NwYWNl IGl0IGlzIG9mZmVyaW5nIGFzIFpWT0w/DQo+IA0KPiBhKSBJcyB0aGlzIHJlYWxseSB0cnVlPw0K PiBiKSBTaG91bGQgSSBoYXZlIGRvbmUgc29tZXRoaW5nIGRpZmZlcmVudCB0byBub3Qgd2FzdGUg c28gbXVjaCBvdmVyaGVhZD8NCj4gDQo+IE5vdGUgdGhhdCB0aGUgY29tcHJlc3Npb24gcmF0ZSBp cyAxLjAweCwgd2hpY2ggaXMgb2YgY291cnNlIGR1ZSB0byANCj4gd3JpdGluZyBoMjY0IG1lZGlh IHN0cmVhbXMgdGhhdCBkbyBub3QgY29tcHJlc3MgYXQgYWxsLiBCdXQgaXMgdGhlIA0KPiBwYXJl bnRzIGRlZmF1bHQsIGFuZCBJIGhhdmVuJ3QgdHVybmVkIGl0IG9mZi4NCj4gDQo+IDIpDQo+IFRo ZSBzZWNvbmQgb25lIG1pZ2h0IGJlIG1vcmUgYSBXaW5kb3dzIHF1ZXN0aW9uLCBidXQgYW55d2F5 cy4NCj4gDQo+IFRoZSBleHBvcnQgaXMgdGhhdCBzYW1lIDVUIFpWT0wsIHBsYWluIGNyZWF0ZSB3 aXRoDQo+IAl6ZnMgY3JlYXRlIC1zIC1WIDVUIHpmc3JhaWQvbnZyMg0KPiBVbmRlciBXaW5kb3dz IEkgdXNlZCB0aGUgcmVndWxhciBzdHVmZiB0byBmb3JtYXQgdGhlIGRpcyB3aXRoIEdQVCBhbmQg DQo+IE5URlMgd2l0aCA4ayBzZWdtZW50cy4gKG1hdGNoaW5nIHRoZSBaVk9MIGJsb2Nrc2l6ZSkN Cj4gDQo+IFVwb24gcmVib290IEZyZWVCU0Qgbm90aWNlcyB0aGUgZm9sbG93aW5nOg0KPiBHRU9N X1BBUlQ6IHBhcnRpdGlvbiAxIG9uICh6dm9sL3pmc3JhaWQvbnZyMiwgR1BUKSBpcyBub3QgYWxp Z25lZCBvbg0KPiAJODE5MiBieXRlcwkNCj4gQnkgaXRzZWxmIG5vdCB0ZWNobmljYWwgcHJvYmxl bS4gQnV0IHRoZSB3YXJuaW5nIGRvZXMgaGludCB0aGF0IHRoZSANCj4gYWxpZ25tZW50IGlzIHRo dXMsIHRoYXQgcGVyZm9ybWFuY2UgbWlnaHQgc3VmZmVyLiB0aGUgYmFja2luZyBkaXNrcyBhcmUg DQo+IFdEIFJFRHMgd2hpY2ggYXJlIDRLIHNlY3RvcnMuLi4uIFNvIHRoZSBtaXNhbGlnbm1lbnQg Y291bGQgY2F1c2UgdG9vIA0KPiBtYW55IHJlYWQvd3JpdGVzIGZvciB3cml0aW5nLg0KPiANCj4g RG9lcyBhbnlib2R5IGhhdmUgYSBjbHVlIGFzIGhvdyB0byBnZXQgV2luZG93cyB0byBkbyB0aGUg YWxpZ25pbmcgY29ycmVjdGx5Pw0KPiANCj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX18NCj4gZnJlZWJzZC1mc0BmcmVlYnNkLm9yZyBtYWlsaW5nIGxpc3QN Cj4gaHR0cHM6Ly9saXN0cy5mcmVlYnNkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2ZyZWVic2QtZnMN Cj4gVG8gdW5zdWJzY3JpYmUsIHNlbmQgYW55IG1haWwgdG8gImZyZWVic2QtZnMtdW5zdWJzY3Jp YmVAZnJlZWJzZC5vcmciDQoNCg0KSXMgaXQgdGhpcyBlZmZlY3QgeW91IGFyZSBzZWVpbmc6DQpo dHRwczovL2ZvcnVtcy5mcmVlYnNkLm9yZy90aHJlYWRzL21pdGlnYXRpbmctNGstZGlzay1pc3N1 ZXMtb24tcmFpZHouMzczNjUvDQoNCi9LDQo= From owner-freebsd-fs@freebsd.org Fri Sep 25 12:36:04 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B368DA090A1 for ; Fri, 25 Sep 2015 12:36:04 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-oi0-x234.google.com (mail-oi0-x234.google.com [IPv6:2607:f8b0:4003:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F8B31C07 for ; Fri, 25 Sep 2015 12:36:04 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by oibi136 with SMTP id i136so57570489oib.3 for ; Fri, 25 Sep 2015 05:36:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=EmeLhauvHMztY47NbZV+IHVDdtqkRRPDAWmvsb6/tSs=; b=NHoO82OgV9QIb23oxFFWIw9RRXXWiCt6pVhGTzTIcJeCHzt7rqSvAyFui0zV/wNJqN nRBFTZOXtg/OayxxvZEAIIzo5O//g9TO3vieQULpfdV5Z4WaCVZ1OBpZc48l/3P4BaTn ViHCjgmh93JeSpvazPdPnelHersb4Dt3TOafeu7DQY0s2otZbx00zQEZs5g0L24kKbth KE7/9RWxTFU43gi/yQTaLIOfyGGF9QJFcEuSo9KMVkzIZnU1yNGj9UUamwQRA4goHUb/ nhUuJrTQoIhn16LIhKPlOY3Gb+CrUHL6IxWAOSrWTy4wUCEDNZW4tJAk1lCOUSl050Dg rpag== MIME-Version: 1.0 X-Received: by 10.202.64.68 with SMTP id n65mr2723895oia.53.1443184563776; Fri, 25 Sep 2015 05:36:03 -0700 (PDT) Received: by 10.202.190.195 with HTTP; Fri, 25 Sep 2015 05:36:03 -0700 (PDT) Date: Fri, 25 Sep 2015 14:36:03 +0200 Message-ID: Subject: ZFS, iSCSI & smartctl From: Ben RUBSON To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Sep 2015 12:36:04 -0000 Hello, Every documentation I read about ZFS indicates that ZFS likes to see the disks directly, so without any RAID adapter etc... I have server_1 with disks plugged to a SAS card, as JBOD. Perfect. However, I would like to use these disks from server_2 through iSCSI. Will ZFS on server_2 like these disks as much as if they were directly used from server_1 ? Will it have all its capabilities through iSCSI, will it see all the disks' details it needs ? Second point, on each diskid (/dev/diskid/...) is created a GPT label. diskid are given to the iSCSI target configuration, so that on the iSCSI initiator are seen the GPT labels. GPT labels are then used on the iSCSI initiator to create a zpool (mirror with disks from the initiator too). When iSCSI target service starts, GPT labels are no more seen from the iSCSI target, diskid however still are. Then, I would like to know, from which server do I have to run smartctl on these disks ? >From the iSCSI target ? >From the iSCSI initiator ? Giving diskid (/dev/diskid/...) to smartctl ? Thank you very much for your help ! Ben From owner-freebsd-fs@freebsd.org Fri Sep 25 14:05:50 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77746A0879B for ; Fri, 25 Sep 2015 14:05:50 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: from mail-ig0-x22d.google.com (mail-ig0-x22d.google.com [IPv6:2607:f8b0:4001:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 421F81F71 for ; Fri, 25 Sep 2015 14:05:50 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: by igcpb10 with SMTP id pb10so12235404igc.1 for ; Fri, 25 Sep 2015 07:05:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=csL/VHuld80mfxArvyDd3XIE/4eC8DSHbWLPgVF/Hyo=; b=Pa3BaHd45RxDsrNkEwKPDMOD+kpWcBTLiv+HePKeGAItgJjQcd6R4ZTIazdhyT1AOs yE7ZbOf5UPfL7nBd86w2lKE/oZYDldR9i1ihhj/JQTroT6nIDzvYdeVcNLE8k9ySi8Em fwhQkvLtPOK/aJSGlaSHjj6GXAH4pfA2+svqMk+s0UYNRjM47XanhhlFrYP90Jp03Ukz L+YEuGVutDPtX3bxBuwbmCga6QVrm75K18PyqkLji9SVf08vJtJ+jKhtLMHaEPHDO051 CnJOJN8JUhXVi6Aa+Cl34j/kvcMi9eihZPcdUQHpDTIao56jU6dkbBIV561lZZbTBpjT avTg== MIME-Version: 1.0 X-Received: by 10.50.134.161 with SMTP id pl1mr2928975igb.28.1443189949525; Fri, 25 Sep 2015 07:05:49 -0700 (PDT) Received: by 10.107.4.72 with HTTP; Fri, 25 Sep 2015 07:05:49 -0700 (PDT) In-Reply-To: <1443179614.5271.42.camel@data-b104.adm.slu.se> References: <56050EFC.1000303@digiware.nl> <1443179614.5271.42.camel@data-b104.adm.slu.se> Date: Fri, 25 Sep 2015 10:05:49 -0400 Message-ID: Subject: Re: ZFS, Zvol, iSCSI and windows From: Tom Curry To: Willem Jan Withagen Cc: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Sep 2015 14:05:50 -0000 On Fri, Sep 25, 2015 at 7:13 AM, Karli Sj=C3=B6berg = wrote: > fre 2015-09-25 klockan 11:08 +0200 skrev Willem Jan Withagen: > > Hi, > > > > Because of the Network Video Recorder (on windows) we use only likes > > "real" disks, and not SMB disks. We started using ZVOLs which are > > exported thru iSCSI/ctld.... > > > > And that works really well, so there all thumbs up for this combo. > > > > However..... (you knew that was coming) > > I do have some questions, and hope somebody can share some insights. > > > > 1) > > This is a ZFS question. > > > > I have created a ZVOL with: > > zfs create -s -V 5T zfsraid/nvr2 > > > > Looking at the disk usage in 'zfs get all': > > NAME PROPERTY VALUE SOURCE > > zfsraid/nvr2 used 9.97T - > > zfsraid/nvr2 available 438G - > > zfsraid/nvr2 referenced 9.97T - > > zfsraid/nvr2 compressratio 1.00x - > > zfsraid/nvr2 reservation none default > > zfsraid/nvr2 volsize 5T local > > zfsraid/nvr2 volblocksize 8K - > > zfsraid/nvr2 checksum on default > > zfsraid/nvr2 compression lz4 default > > zfsraid/nvr2 primarycache all default > > zfsraid/nvr2 secondarycache all default > > zfsraid/nvr2 usedbysnapshots 0 - > > zfsraid/nvr2 usedbydataset 9.97T - > > zfsraid/nvr2 usedbychildren 0 - > > zfsraid/nvr2 usedbyrefreservation 0 - > > zfsraid/nvr2 sync standard default > > zfsraid/nvr2 written 9.97T - > > zfsraid/nvr2 logicalused 4.97T - > > zfsraid/nvr2 logicalreferenced 4.97T - > > zfsraid/nvr2 volmode default default > > > > And what sort of "worries" me is that it seems that this volume is usin= g > > twice the amount of diskspace it is offering as ZVOL? > > > > a) Is this really true? > > b) Should I have done something different to not waste so much overhead= ? > > > > Note that the compression rate is 1.00x, which is of course due to > > writing h264 media streams that do not compress at all. But is the > > parents default, and I haven't turned it off. > > > > 2) > > The second one might be more a Windows question, but anyways. > > > > The export is that same 5T ZVOL, plain create with > > zfs create -s -V 5T zfsraid/nvr2 > > Under Windows I used the regular stuff to format the dis with GPT and > > NTFS with 8k segments. (matching the ZVOL blocksize) > > > > Upon reboot FreeBSD notices the following: > > GEOM_PART: partition 1 on (zvol/zfsraid/nvr2, GPT) is not aligned on > > 8192 bytes > > By itself not technical problem. But the warning does hint that the > > alignment is thus, that performance might suffer. the backing disks are > > WD REDs which are 4K sectors.... So the misalignment could cause too > > many read/writes for writing. > > > > Does anybody have a clue as how to get Windows to do the aligning > correctly? > > > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > Is it this effect you are seeing: > > https://forums.freebsd.org/threads/mitigating-4k-disk-issues-on-raidz.373= 65/ > > /K > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > I ran into the same problem, the solution was to create the zvol with a larger volblocksize like 32k. If space efficiency is the primary concern I believe you need to choose a blocksize that is greater than or equal to the number of non parity drives multiplied by their sector size. Judging by your overhead I'm going to assume you have a 6 disk raidz2, so 4 * 4k =3D 1= 6k at least. Try creating another zvol with "-o volblocksize=3D16k" and see ho= w that treats you. As to your second question, modern versions of windows should align partitions correctly, what version are you using? You may wish to open an elevated command prompt and sanity check the output of "fsutil fsinfo ntfsinfo x:" Finally, from my own experience with using iscsi under windows I found I had a random 4k iops increase from ~10,000 to ~90,000 (which is close to my pools real speed) by adjusting the following registry key Create this DWORD key with a value of 1: HKLM\SYSTEM\CurrentControlSet\Control\Class\{4D36E97B-E325-11CE-BFC1-08002B= E10318}\\Parameters\iSCSIDisableNagle From owner-freebsd-fs@freebsd.org Sat Sep 26 03:52:42 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E0B5A093C4 for ; Sat, 26 Sep 2015 03:52:42 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (vps.rulingia.com [103.243.244.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps.rulingia.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E29CB1043 for ; Sat, 26 Sep 2015 03:52:40 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (c220-239-242-83.belrs5.nsw.optusnet.com.au [220.239.242.83]) by vps.rulingia.com (8.15.2/8.15.2) with ESMTPS id t8Q3qRdG083987 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 26 Sep 2015 13:52:33 +1000 (AEST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id t8Q3qK8U099159 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 26 Sep 2015 13:52:20 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id t8Q3qI5F099155; Sat, 26 Sep 2015 13:52:18 +1000 (AEST) (envelope-from peter) Date: Sat, 26 Sep 2015 13:52:18 +1000 From: Peter Jeremy To: Pekka =?iso-8859-1?Q?J=E4rvinen?= Cc: freebsd-fs@freebsd.org Subject: Re: Missing free space from new raidz2 zpool Message-ID: <20150926035218.GF3478@server.rulingia.com> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="vmttodhTwj0NAgWp" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender succeeded STARTTLS authentication, not delayed by milter-greylist-4.4.3 (vps.rulingia.com [103.243.244.15]); Sat, 26 Sep 2015 13:52:35 +1000 (AEST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Sep 2015 03:52:42 -0000 --vmttodhTwj0NAgWp Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2015-Sep-24 19:36:50 +0300, Pekka J=E4rvinen = wrote: >I bought 6 new 8 TB drives and created new raidz2 pool with these drives. > >My question is why zfs list and df is only showing 14.5T free? Shouldn't it >be more close to 30T? Or is that old zfs metadata somehow lurking around >and zfs is reading that? Bug? Your pool seems to be sized correctly but the filesystem can't see all the space. If you're concerned about old metadata, destroy your new pool and then dd zeroes over the first and last few MB of each drive - which is where ZFS stores its metadata: for i in /dev/ada{0,1,2,3,4,5};do dd if=3D/dev/zero of=3D$i bs=3D64k count=3D32 dd if=3D/dev/zero of=3D$i bs=3D64k oseek=3D122094000 done ># zpool iostat storage2 > capacity operations bandwidth >pool alloc free read write read write >---------- ----- ----- ----- ----- ----- ----- >storage2 900K 43.5T 0 1 0 7.84K The 43.5T looks correct. "zpool list" would be a better command. ># zdb storage2 > >Cached configuration: =2E.. > asize: 48009350479872 =2E.. >MOS Configuration: =2E.. > asize: 48009350479872 That's also correct. As an experiment, I simulated your configuration: # cd /tmp # mkdir zfs # cd zfs # for i in 0 1 2 3 4 5;do truncate -s 7814026584k d$i;done # zpool create storage2 raidz2 /tmp/zfs/d? # zpool list storage2 NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTRO= OT storage2 43.5T 165K 43.5T - 0% 0% 1.00x ONLINE - # zfs list storage2 NAME USED AVAIL REFER MOUNTPOINT storage2 88.9K 28.1T 32.0K /storage2 # zfs get all storage2 NAME PROPERTY VALUE SOURCE storage2 type filesystem - storage2 creation Sat Sep 26 13:31 2015 - storage2 used 88.9K - storage2 available 28.1T - storage2 referenced 32.0K - storage2 compressratio 1.00x - storage2 mounted yes - =2E.. --=20 Peter Jeremy --vmttodhTwj0NAgWp Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJWBhZyXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFRUIyOTg2QzMwNjcxRTc0RTY1QzIyN0Ux NkE1OTdBMEU0QTIwQjM0AAoJEBall6Dkogs0C9cP/2/PRbu44/LpFScWnvWEvQe/ MHsuJqdw4vnQkZj/F+Jf+XYFh+CMVOpi6/2PLdMr40aC/Iw6EkKR6+EQDVHR4qGt al9YC7ctCRcyHfwE5hbQvjibQFpI7nWDaT9j9ZsyiIMtMdprV9LyyT/qjkHNo4AM f/iNJ0A9VZXgPxfIX309dO4z62SfVvr8p+M3FKHB/KjFQXeebfQr87bjhDyObs/5 jsmbjiTWJaWqIMpkf+vfL/YmHRy89Z1Ya5TJKcioRkiUgZ0dcRW0lUtaU1Yy+HiU p4id/hwrFmmlbCGwmWNipg5F5+3iRI1A0mZlF0pecQJnPSw2PX31s/FTNWUfweLL KgA+/abqA+24w30jhPkHN8uBzAAFbmtK+Z1jFPoA+i6BDPzQJSSgqGmhgdIoqCB6 txMg9aIzgmyl9i7q7g6S1dT+3Yo+n7XlIn9bw/jf62bZ/buWdRtxduVy4/tVD2/C 6/QaRou37oENBPLaTewcagfCnifLLkBnpw+SJfJd3IVyjN4huH0FLyPY2xnVB+w9 zWPu2Y+e/aqr9n1Tis3qmBoGa/t4JLXuZyCg2A+rpuzOyyoJCcPGZUVgrsaQqL5X +J+IuXoQ+U0DFWeiKDmmT2hbgtfGLlAntqoQ43NJ7dyriD+gr2O8lAbQahU3sOBj +pSn2KZJ74cusZplPw7N =x5vu -----END PGP SIGNATURE----- --vmttodhTwj0NAgWp-- From owner-freebsd-fs@freebsd.org Sat Sep 26 05:43:27 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0131CA08F00 for ; Sat, 26 Sep 2015 05:43:27 +0000 (UTC) (envelope-from pekka.jarvinen@gmail.com) Received: from mail-io0-x235.google.com (mail-io0-x235.google.com [IPv6:2607:f8b0:4001:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D0B16190B for ; Sat, 26 Sep 2015 05:43:26 +0000 (UTC) (envelope-from pekka.jarvinen@gmail.com) Received: by iofh134 with SMTP id h134so130685266iof.0 for ; Fri, 25 Sep 2015 22:43:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=CbcxEKmvlPp8yGQanG+/TCqBFIBD282Br1GPnkyvUUY=; b=vcdM8Wf+Z1U9mRIKHesWSZMCUwDxTy7Jf9Fk50QcKjCMqDbBPdm7Dnxre4ZrOymJtT xUzBHg2luvxDnW/imaKar+510Y9d8R0Fu61oHAwgHVnNwr1CMIBiTD2cEO3ZeqpXirN+ uVoTkdprDMgIJ72MRsRcao4/JsFzLxN1zuB/zav9mfPe+dugmdwz46rxIed/G8Ktb/Ng MplvjxyjFXRFx7s6jS7Q6lhqQqQrIdWJDIkGny1CLRU7ttqxZ7+BrbrgnQTW4mTagm86 1k33QCDLyYc9oMQuTwPV501MQHLzNytDVtGCGSg9u20lsSSonqglQtfFW6IVUQw+ndEt 6IDQ== X-Received: by 10.107.30.12 with SMTP id e12mr9644298ioe.57.1443246206093; Fri, 25 Sep 2015 22:43:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.92.6 with HTTP; Fri, 25 Sep 2015 22:42:56 -0700 (PDT) In-Reply-To: <20150926035218.GF3478@server.rulingia.com> References: <20150926035218.GF3478@server.rulingia.com> From: =?UTF-8?Q?Pekka_J=C3=A4rvinen?= Date: Sat, 26 Sep 2015 08:42:56 +0300 Message-ID: Subject: Re: Missing free space from new raidz2 zpool To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Sep 2015 05:43:27 -0000 I installed FreeBSD-10.2-STABLE-amd64-20150917-r287929-memstick.img and it gives 28.1T free. So possibly a bug in FreeBSD 10.2-RELEASE-p2 #0 r287260M. I created bug report https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203330 I also tried with OpenIndiana's ZFS implementation with same machine and it gives 28.5T free and lists drives as 8001.49GB. 2015-09-26 6:52 GMT+03:00 Peter Jeremy : > On 2015-Sep-24 19:36:50 +0300, Pekka J=C3=A4rvinen > wrote: > >I bought 6 new 8 TB drives and created new raidz2 pool with these drives= . > > > >My question is why zfs list and df is only showing 14.5T free? Shouldn't > it > >be more close to 30T? Or is that old zfs metadata somehow lurking around > >and zfs is reading that? Bug? > > Your pool seems to be sized correctly but the filesystem can't see all > the space. If you're concerned about old metadata, destroy your new > pool and then dd zeroes over the first and last few MB of each drive - > which is where ZFS stores its metadata: > > for i in /dev/ada{0,1,2,3,4,5};do > dd if=3D/dev/zero of=3D$i bs=3D64k count=3D32 > dd if=3D/dev/zero of=3D$i bs=3D64k oseek=3D122094000 > done > > ># zpool iostat storage2 > > capacity operations bandwidth > >pool alloc free read write read write > >---------- ----- ----- ----- ----- ----- ----- > >storage2 900K 43.5T 0 1 0 7.84K > > The 43.5T looks correct. "zpool list" would be a better command. > > ># zdb storage2 > > > >Cached configuration: > ... > > asize: 48009350479872 > ... > >MOS Configuration: > ... > > asize: 48009350479872 > > That's also correct. > > As an experiment, I simulated your configuration: > # cd /tmp > # mkdir zfs > # cd zfs > # for i in 0 1 2 3 4 5;do truncate -s 7814026584k d$i;done > # zpool create storage2 raidz2 /tmp/zfs/d? > # zpool list storage2 > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH > ALTROOT > storage2 43.5T 165K 43.5T - 0% 0% 1.00x ONLINE - > # zfs list storage2 > NAME USED AVAIL REFER MOUNTPOINT > storage2 88.9K 28.1T 32.0K /storage2 > # zfs get all storage2 > NAME PROPERTY VALUE SOURCE > storage2 type filesystem - > storage2 creation Sat Sep 26 13:31 2015 - > storage2 used 88.9K - > storage2 available 28.1T - > storage2 referenced 32.0K - > storage2 compressratio 1.00x - > storage2 mounted yes - > ... > > -- > Peter Jeremy > --=20 Pekka J=C3=A4rvinen From owner-freebsd-fs@freebsd.org Sat Sep 26 17:29:16 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DFE9CA083B7 for ; Sat, 26 Sep 2015 17:29:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD36CDB5 for ; Sat, 26 Sep 2015 17:29:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8QHTGHf030387 for ; Sat, 26 Sep 2015 17:29:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 203330] ZFS free space bug or HDD driver bug Date: Sat, 26 Sep 2015 17:29:16 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Sep 2015 17:29:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203330 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@freebsd.org Sat Sep 26 19:03:25 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F7D8A0A6D2 for ; Sat, 26 Sep 2015 19:03:25 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E638B119 for ; Sat, 26 Sep 2015 19:03:24 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 17F54153431; Sat, 26 Sep 2015 21:03:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ED4xObNkxf6s; Sat, 26 Sep 2015 21:03:07 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:953f:f3db:1564:452f] (unknown [IPv6:2001:4cb8:3:1:953f:f3db:1564:452f]) by smtp.digiware.nl (Postfix) with ESMTPA id 8F5D6153430; Sat, 26 Sep 2015 21:03:07 +0200 (CEST) Subject: Re: ZFS, Zvol, iSCSI and windows To: Tom Curry References: <56050EFC.1000303@digiware.nl> <1443179614.5271.42.camel@data-b104.adm.slu.se> Cc: "freebsd-fs@freebsd.org" From: Willem Jan Withagen Message-ID: <5606EBEA.2090103@digiware.nl> Date: Sat, 26 Sep 2015 21:03:06 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Sep 2015 19:03:25 -0000 On 25-9-2015 16:05, Tom Curry wrote: > On Fri, Sep 25, 2015 at 7:13 AM, Karli Sjöberg > wrote: > > fre 2015-09-25 klockan 11:08 +0200 skrev Willem Jan Withagen: > > Hi, > > > > Because of the Network Video Recorder (on windows) we use only likes > > "real" disks, and not SMB disks. We started using ZVOLs which are > > exported thru iSCSI/ctld.... > > > > And that works really well, so there all thumbs up for this combo. > > > > However..... (you knew that was coming) > > I do have some questions, and hope somebody can share some insights. > > > > 1) > > This is a ZFS question. > > > > I have created a ZVOL with: > > zfs create -s -V 5T zfsraid/nvr2 > > > > Looking at the disk usage in 'zfs get all': > > NAME PROPERTY VALUE SOURCE > > zfsraid/nvr2 used 9.97T - > > zfsraid/nvr2 available 438G - > > zfsraid/nvr2 referenced 9.97T - > > zfsraid/nvr2 compressratio 1.00x - > > zfsraid/nvr2 reservation none default > > zfsraid/nvr2 volsize 5T local > > zfsraid/nvr2 volblocksize 8K - > > zfsraid/nvr2 checksum on default > > zfsraid/nvr2 compression lz4 default > > zfsraid/nvr2 primarycache all default > > zfsraid/nvr2 secondarycache all default > > zfsraid/nvr2 usedbysnapshots 0 - > > zfsraid/nvr2 usedbydataset 9.97T - > > zfsraid/nvr2 usedbychildren 0 - > > zfsraid/nvr2 usedbyrefreservation 0 - > > zfsraid/nvr2 sync standard default > > zfsraid/nvr2 written 9.97T - > > zfsraid/nvr2 logicalused 4.97T - > > zfsraid/nvr2 logicalreferenced 4.97T - > > zfsraid/nvr2 volmode default default > > > > And what sort of "worries" me is that it seems that this volume is > using > > twice the amount of diskspace it is offering as ZVOL? > > > > a) Is this really true? > > b) Should I have done something different to not waste so much > overhead? > > > > Note that the compression rate is 1.00x, which is of course due to > > writing h264 media streams that do not compress at all. But is the > > parents default, and I haven't turned it off. > > > > 2) > > The second one might be more a Windows question, but anyways. > > > > The export is that same 5T ZVOL, plain create with > > zfs create -s -V 5T zfsraid/nvr2 > > Under Windows I used the regular stuff to format the dis with GPT and > > NTFS with 8k segments. (matching the ZVOL blocksize) > > > > Upon reboot FreeBSD notices the following: > > GEOM_PART: partition 1 on (zvol/zfsraid/nvr2, GPT) is not aligned on > > 8192 bytes > > By itself not technical problem. But the warning does hint that the > > alignment is thus, that performance might suffer. the backing > disks are > > WD REDs which are 4K sectors.... So the misalignment could cause too > > many read/writes for writing. > > > > Does anybody have a clue as how to get Windows to do the aligning > correctly? > > > Is it this effect you are seeing: > https://forums.freebsd.org/threads/mitigating-4k-disk-issues-on-raidz.37365/ The article attributes certain issues to using 4k blocks zvols in zraid. It could very well be on the money. But it seems testing is the best way to find out. > > I ran into the same problem, the solution was to create the zvol with a > larger volblocksize like 32k. If space efficiency is the primary concern > I believe you need to choose a blocksize that is greater than or equal > to the number of non parity drives multiplied by their sector size. > Judging by your overhead I'm going to assume you have a 6 disk raidz2, > so 4 * 4k = 16k at least. Try creating another zvol with "-o > volblocksize=16k" and see how that treats you. Right guess, Actually the it is a 2* 6 vol raidz2 THe article seems to show that the overhead gets less and less when the blocksize increases. In which case I start worrying about the performance, because writting smaller blocks will require a read/wite cycle. In my specific case I'm writing large video files (5Gb) in streaming mode, so the read/write effect will be mitigated due to streaming. I'll go and make a few ZVOLS and see what happens with the overhead when they start filling up. But it looks like the windows blocksize should certainly not match zvol blocksize. > As to your second question, modern versions of windows should align > partitions correctly, what version are you using? You may wish to open > an elevated command prompt and sanity check the output of "fsutil fsinfo > ntfsinfo x:" I'm just loading the disk in explorer, rightclick it, and then format. Just as basic as that. Not sure what to make of it, but this gives: C:\WINDOWS\system32>fsutil fsinfo ntfsinfo m: NTFS Volume Serial Number : 0x2aaa50b8aa50826d NTFS Version : 3.1 LFS Version : 2.0 Number Sectors : 0x000000027ffbefff Total Clusters : 0x0000000027ffbeff Free Clusters : 0x00000000008497fc Total Reserved : 0x00000000000a8ec0 Bytes Per Sector : 512 Bytes Per Physical Sector : 4096 Bytes Per Cluster : 8192 Bytes Per FileRecord Segment : 1024 Clusters Per FileRecord Segment : 0 Mft Valid Data Length : 0x0000000000140000 Mft Start Lcn : 0x0000000000060000 Mft2 Start Lcn : 0x0000000000000001 Mft Zone Start : 0x00000000000600a0 Mft Zone End : 0x0000000000066420 Max Device Trim Extent Count : 4294967295 Max Device Trim Byte Count : 0xffffffff Max Volume Trim Extent Count : 62 Max Volume Trim Byte Count : 0x40000000 Resource Manager Identifier : FD836E1F-1372-11E5-825F-000272131F3C The gpart info in the zvol is reported as unaligned when booting FreeBSD, but that would be because windows put it in there in an unaligned postition. Something like at 63 * 512 bytes...?? Any commandline incarnation that allows me to put it a a different offset? > > Finally, from my own experience with using iscsi under windows I > found I had a random 4k iops increase from ~10,000 to ~90,000 (which > is close to my pools real speed) by adjusting the following registry > key > Create this DWORD key with a value of 1: > HKLM\SYSTEM\CurrentControlSet\Control\Class\{4D36E97B-E325-11CE-BFC1-08002BE10318}\ Number>\Parameters\iSCSIDisableNagle Very usefull hint, thanx for that. --WjW From owner-freebsd-fs@freebsd.org Sat Sep 26 20:31:02 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0FB11A08A88 for ; Sat, 26 Sep 2015 20:31:02 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from Exch2-3.slu.se (exch2-3.slu.se [77.235.224.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56D4D155 for ; Sat, 26 Sep 2015 20:31:00 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by Exch2-3.slu.se (77.235.224.123) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Sat, 26 Sep 2015 22:30:50 +0200 Received: from exch2-4.slu.se ([fe80::bc24:95e8:1e1e:f3ae]) by exch2-4.slu.se ([fe80::bc24:95e8:1e1e:f3ae%22]) with mapi id 15.00.1104.000; Sat, 26 Sep 2015 22:30:50 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Willem Jan Withagen CC: "freebsd-fs@freebsd.org" , Tom Curry Subject: Re: ZFS, Zvol, iSCSI and windows Thread-Topic: ZFS, Zvol, iSCSI and windows Thread-Index: AQHQ+Jo7AK1gWL7n30CZVpf4MzfBnA== Date: Sat, 26 Sep 2015 20:30:49 +0000 Message-ID: Accept-Language: sv-SE, en-US Content-Language: sv-SE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Sep 2015 20:31:02 -0000 DQpEZW4gMjYgc2VwIDIwMTUgOTowMyBlbSBza3JldiBXaWxsZW0gSmFuIFdpdGhhZ2VuIDx3andA ZGlnaXdhcmUubmw+Og0KPg0KPiBPbiAyNS05LTIwMTUgMTY6MDUsIFRvbSBDdXJyeSB3cm90ZToN Cj4gPiBPbiBGcmksIFNlcCAyNSwgMjAxNSBhdCA3OjEzIEFNLCBLYXJsaSBTasO2YmVyZyA8a2Fy bGkuc2pvYmVyZ0BzbHUuc2UNCj4gPiA8bWFpbHRvOmthcmxpLnNqb2JlcmdAc2x1LnNlPj4gd3Jv dGU6DQo+ID4NCj4gPiAgICAgZnJlIDIwMTUtMDktMjUga2xvY2thbiAxMTowOCArMDIwMCBza3Jl diBXaWxsZW0gSmFuIFdpdGhhZ2VuOg0KPiA+ICAgICA+IEhpLA0KPiA+ICAgICA+DQo+ID4gICAg ID4gQmVjYXVzZSBvZiB0aGUgTmV0d29yayBWaWRlbyBSZWNvcmRlciAob24gd2luZG93cykgd2Ug dXNlIG9ubHkgbGlrZXMNCj4gPiAgICAgPiAicmVhbCIgZGlza3MsIGFuZCBub3QgU01CIGRpc2tz LiBXZSBzdGFydGVkIHVzaW5nIFpWT0xzIHdoaWNoIGFyZQ0KPiA+ICAgICA+IGV4cG9ydGVkIHRo cnUgaVNDU0kvY3RsZC4uLi4NCj4gPiAgICAgPg0KPiA+ICAgICA+IEFuZCB0aGF0IHdvcmtzIHJl YWxseSB3ZWxsLCBzbyB0aGVyZSBhbGwgdGh1bWJzIHVwIGZvciB0aGlzIGNvbWJvLg0KPiA+ICAg ICA+DQo+ID4gICAgID4gSG93ZXZlci4uLi4uICh5b3Uga25ldyB0aGF0IHdhcyBjb21pbmcpDQo+ ID4gICAgID4gSSBkbyBoYXZlIHNvbWUgcXVlc3Rpb25zLCBhbmQgaG9wZSBzb21lYm9keSBjYW4g c2hhcmUgc29tZSBpbnNpZ2h0cy4NCj4gPiAgICAgPg0KPiA+ICAgICA+IDEpDQo+ID4gICAgID4g VGhpcyBpcyBhIFpGUyBxdWVzdGlvbi4NCj4gPiAgICAgPg0KPiA+ICAgICA+IEkgaGF2ZSBjcmVh dGVkIGEgWlZPTCB3aXRoOg0KPiA+ICAgICA+ICAgICAgIHpmcyBjcmVhdGUgLXMgLVYgNVQgemZz cmFpZC9udnIyDQo+ID4gICAgID4NCj4gPiAgICAgPiBMb29raW5nIGF0IHRoZSBkaXNrIHVzYWdl IGluICd6ZnMgZ2V0IGFsbCc6DQo+ID4gICAgID4gTkFNRSAgICAgICAgICBQUk9QRVJUWSAgICAg ICAgICAgICAgVkFMVUUgICAgICAgICAgICAgICAgICBTT1VSQ0UNCj4gPiAgICAgPiB6ZnNyYWlk L252cjIgIHVzZWQgICAgICAgICAgICAgICAgICA5Ljk3VCAgICAgICAgICAgICAgICAgIC0NCj4g PiAgICAgPiB6ZnNyYWlkL252cjIgIGF2YWlsYWJsZSAgICAgICAgICAgICA0MzhHICAgICAgICAg ICAgICAgICAgIC0NCj4gPiAgICAgPiB6ZnNyYWlkL252cjIgIHJlZmVyZW5jZWQgICAgICAgICAg ICA5Ljk3VCAgICAgICAgICAgICAgICAgIC0NCj4gPiAgICAgPiB6ZnNyYWlkL252cjIgIGNvbXBy ZXNzcmF0aW8gICAgICAgICAxLjAweCAgICAgICAgICAgICAgICAgIC0NCj4gPiAgICAgPiB6ZnNy YWlkL252cjIgIHJlc2VydmF0aW9uICAgICAgICAgICBub25lICAgICAgICAgICAgICAgICAgIGRl ZmF1bHQNCj4gPiAgICAgPiB6ZnNyYWlkL252cjIgIHZvbHNpemUgICAgICAgICAgICAgICA1VCAg ICAgICAgICAgICAgICAgICAgIGxvY2FsDQo+ID4gICAgID4gemZzcmFpZC9udnIyICB2b2xibG9j a3NpemUgICAgICAgICAgOEsgICAgICAgICAgICAgICAgICAgICAtDQo+ID4gICAgID4gemZzcmFp ZC9udnIyICBjaGVja3N1bSAgICAgICAgICAgICAgb24gICAgICAgICAgICAgICAgICAgICBkZWZh dWx0DQo+ID4gICAgID4gemZzcmFpZC9udnIyICBjb21wcmVzc2lvbiAgICAgICAgICAgbHo0ICAg ICAgICAgICAgICAgICAgICBkZWZhdWx0DQo+ID4gICAgID4gemZzcmFpZC9udnIyICBwcmltYXJ5 Y2FjaGUgICAgICAgICAgYWxsICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQo+ID4gICAgID4g emZzcmFpZC9udnIyICBzZWNvbmRhcnljYWNoZSAgICAgICAgYWxsICAgICAgICAgICAgICAgICAg ICBkZWZhdWx0DQo+ID4gICAgID4gemZzcmFpZC9udnIyICB1c2VkYnlzbmFwc2hvdHMgICAgICAg MCAgICAgICAgICAgICAgICAgICAgICAtDQo+ID4gICAgID4gemZzcmFpZC9udnIyICB1c2VkYnlk YXRhc2V0ICAgICAgICAgOS45N1QgICAgICAgICAgICAgICAgICAtDQo+ID4gICAgID4gemZzcmFp ZC9udnIyICB1c2VkYnljaGlsZHJlbiAgICAgICAgMCAgICAgICAgICAgICAgICAgICAgICAtDQo+ ID4gICAgID4gemZzcmFpZC9udnIyICB1c2VkYnlyZWZyZXNlcnZhdGlvbiAgMCAgICAgICAgICAg ICAgICAgICAgICAtDQo+ID4gICAgID4gemZzcmFpZC9udnIyICBzeW5jICAgICAgICAgICAgICAg ICAgc3RhbmRhcmQgICAgICAgICAgICAgICBkZWZhdWx0DQo+ID4gICAgID4gemZzcmFpZC9udnIy ICB3cml0dGVuICAgICAgICAgICAgICAgOS45N1QgICAgICAgICAgICAgICAgICAtDQo+ID4gICAg ID4gemZzcmFpZC9udnIyICBsb2dpY2FsdXNlZCAgICAgICAgICAgNC45N1QgICAgICAgICAgICAg ICAgICAtDQo+ID4gICAgID4gemZzcmFpZC9udnIyICBsb2dpY2FscmVmZXJlbmNlZCAgICAgNC45 N1QgICAgICAgICAgICAgICAgICAtDQo+ID4gICAgID4gemZzcmFpZC9udnIyICB2b2xtb2RlICAg ICAgICAgICAgICAgZGVmYXVsdCAgICAgICAgICAgICAgICBkZWZhdWx0DQo+ID4gICAgID4NCj4g PiAgICAgPiBBbmQgd2hhdCBzb3J0IG9mICJ3b3JyaWVzIiBtZSBpcyB0aGF0IGl0IHNlZW1zIHRo YXQgdGhpcyB2b2x1bWUgaXMNCj4gPiAgICAgdXNpbmcNCj4gPiAgICAgPiB0d2ljZSB0aGUgYW1v dW50IG9mIGRpc2tzcGFjZSBpdCBpcyBvZmZlcmluZyBhcyBaVk9MPw0KPiA+ICAgICA+DQo+ID4g ICAgID4gYSkgSXMgdGhpcyByZWFsbHkgdHJ1ZT8NCj4gPiAgICAgPiBiKSBTaG91bGQgSSBoYXZl IGRvbmUgc29tZXRoaW5nIGRpZmZlcmVudCB0byBub3Qgd2FzdGUgc28gbXVjaA0KPiA+ICAgICBv dmVyaGVhZD8NCj4gPiAgICAgPg0KPiA+ICAgICA+IE5vdGUgdGhhdCB0aGUgY29tcHJlc3Npb24g cmF0ZSBpcyAxLjAweCwgd2hpY2ggaXMgb2YgY291cnNlIGR1ZSB0bw0KPiA+ICAgICA+IHdyaXRp bmcgaDI2NCBtZWRpYSBzdHJlYW1zIHRoYXQgZG8gbm90IGNvbXByZXNzIGF0IGFsbC4gQnV0IGlz IHRoZQ0KPiA+ICAgICA+IHBhcmVudHMgZGVmYXVsdCwgYW5kIEkgaGF2ZW4ndCB0dXJuZWQgaXQg b2ZmLg0KPiA+ICAgICA+DQo+ID4gICAgID4gMikNCj4gPiAgICAgPiBUaGUgc2Vjb25kIG9uZSBt aWdodCBiZSBtb3JlIGEgV2luZG93cyBxdWVzdGlvbiwgYnV0IGFueXdheXMuDQo+ID4gICAgID4N Cj4gPiAgICAgPiBUaGUgZXhwb3J0IGlzIHRoYXQgc2FtZSA1VCBaVk9MLCBwbGFpbiBjcmVhdGUg d2l0aA0KPiA+ICAgICA+ICAgICAgIHpmcyBjcmVhdGUgLXMgLVYgNVQgemZzcmFpZC9udnIyDQo+ ID4gICAgID4gVW5kZXIgV2luZG93cyBJIHVzZWQgdGhlIHJlZ3VsYXIgc3R1ZmYgdG8gZm9ybWF0 IHRoZSBkaXMgd2l0aCBHUFQgYW5kDQo+ID4gICAgID4gTlRGUyB3aXRoIDhrIHNlZ21lbnRzLiAo bWF0Y2hpbmcgdGhlIFpWT0wgYmxvY2tzaXplKQ0KPiA+ICAgICA+DQo+ID4gICAgID4gVXBvbiBy ZWJvb3QgRnJlZUJTRCBub3RpY2VzIHRoZSBmb2xsb3dpbmc6DQo+ID4gICAgID4gR0VPTV9QQVJU OiBwYXJ0aXRpb24gMSBvbiAoenZvbC96ZnNyYWlkL252cjIsIEdQVCkgaXMgbm90IGFsaWduZWQg b24NCj4gPiAgICAgPiAgICAgICA4MTkyIGJ5dGVzDQo+ID4gICAgID4gQnkgaXRzZWxmIG5vdCB0 ZWNobmljYWwgcHJvYmxlbS4gQnV0IHRoZSB3YXJuaW5nIGRvZXMgaGludCB0aGF0IHRoZQ0KPiA+ ICAgICA+IGFsaWdubWVudCBpcyB0aHVzLCB0aGF0IHBlcmZvcm1hbmNlIG1pZ2h0IHN1ZmZlci4g dGhlIGJhY2tpbmcNCj4gPiAgICAgZGlza3MgYXJlDQo+ID4gICAgID4gV0QgUkVEcyB3aGljaCBh cmUgNEsgc2VjdG9ycy4uLi4gU28gdGhlIG1pc2FsaWdubWVudCBjb3VsZCBjYXVzZSB0b28NCj4g PiAgICAgPiBtYW55IHJlYWQvd3JpdGVzIGZvciB3cml0aW5nLg0KPiA+ICAgICA+DQo+ID4gICAg ID4gRG9lcyBhbnlib2R5IGhhdmUgYSBjbHVlIGFzIGhvdyB0byBnZXQgV2luZG93cyB0byBkbyB0 aGUgYWxpZ25pbmcNCj4gPiAgICAgY29ycmVjdGx5Pw0KPiA+DQo+ID4NCj4gPiAgICAgSXMgaXQg dGhpcyBlZmZlY3QgeW91IGFyZSBzZWVpbmc6DQo+ID4gICAgIGh0dHBzOi8vZm9ydW1zLmZyZWVi c2Qub3JnL3RocmVhZHMvbWl0aWdhdGluZy00ay1kaXNrLWlzc3Vlcy1vbi1yYWlkei4zNzM2NS8N Cj4NCj4gVGhlIGFydGljbGUgYXR0cmlidXRlcyBjZXJ0YWluIGlzc3VlcyB0byB1c2luZyA0ayBi bG9ja3MgenZvbHMgaW4genJhaWQuDQo+IEl0IGNvdWxkIHZlcnkgd2VsbCBiZSBvbiB0aGUgbW9u ZXkuIEJ1dCBpdCBzZWVtcyB0ZXN0aW5nIGlzIHRoZSBiZXN0IHdheQ0KPiB0byBmaW5kIG91dC4N Cg0KSXQgbWF5IGhhdmUgYmVlbiB1bmNsZWFyIGJ1dCB3aGF0IEkgd2FzIHRyeWluZyB0byBzaG93 IGluIG15IHBvc3RzIHdhcyB0aGF0IHdoaWxlIHVzZWQgc3BhY2UgZnJvbSB0aGUgaW5pdGlhdG9y IHBvaW50IG9mIHZpZXcgaXMgYWx3YXlzIHRoZSBzYW1lLCB1c2VkIHNwYWNlIGluIHRoZSBzdG9y YWdlIGRpZmZlcnMgd2lsZGx5IGRlcGVuZGluZyBvbiB0aGUgInZvbGJsb2Nrc2l6ZSIgeW91IGNy ZWF0ZSB0aGUgenZvbHMgd2l0aCBhbmQgdGhlICJBbGxvY2F0aW9uIHVuaXQgc2l6ZSIgeW91IGNo b29zZSB3aGVuIHlvdSBmb3JtYXQgdGhlIHZvbHVtZSB3aXRoaW4gdGhlIGluaXRpYXRvci4gU2Vl bXMgYXMgaWYgeW91IHVzZSBhIHRvbyBzbWFsbCBzaXplLCBaRlMgY2FuJ3Qgc3RyaXBlIHRoZSBk YXRhIGFjcm9zcyB0aGUgdmRldiBhbmQgaXMgZm9yY2VkIHRvIHdyaXRlIG91dCB0aGUgc2FtZSBk YXRhIHRvIGFsbCBkcml2ZXMgaW4gdGhlIHZkZXYuIEF0IGxlYXN0IEkgaG9wZSB0aGUgcHJvYmxl bSBpcyBjb25maW5lZCB0byB0aGUgdmRldiwgYnV0IGNvdWxkIGJlIGl0J3MgcG9vbC13aWRlICh5 aWtlcyEpDQoNCkFueWhvdywgY2hvb3NpbmcgYSBsYXJnZXIgYmxvY2stIGFuZCBhbGxvY2F0aW9u IChjbHVzdGVyKSBzaXplIHNlZW1zIHRvIG1pdGlnYXRlIGl0IHByZXR0eSBlZmZpY2llbnRseSwg c2VlbXMgeW91IGFjdHVhbGx5IGRpZG4ndCBzcGVjaWZ5IHRoYXQgd2hlbiBjcmVhdGluZyB0aGUg enZvbC4gQ291bGQgeW91IGRvICJ6ZnMgZ2V0IHZvbGJsb2Nrc2l6ZSB6ZnNyYWlkL252cjIiIHRv IHNlZSB3aGF0IHdhcyBjaG9zZW4/IDhrPyBBdCBsZWFzdCB0aGF0IHNlZW1zIHRvIGJlIHRoZSBj bHVzdGVyIHNpemUgaW4gdGhlIGZpbGVzeXN0ZW0sIGp1ZGdpbmcgYnkgdGhlIG91dHB1dCB5b3Ug cG9zdGVkIGJlbG93LiBJJ2Qgc3VnZ2VzdCB5b3UgdG8gbWFrZSBhbm90aGVyIG9uZSB3aXRoIDY0 ayB2b2xibG9ja3NpemUvYWxsb2NhdGlvbiB1bml0IHNpemUgYW5kIHNlZSBob3cgdGhhdCB0cmVl dHMgeW91LCBlc3BlY2lhbGx5IHNpbmNlIHlvdSdyZSBkb21pbmFudGx5IHN0b3JpbmcgbGFyZ2Ug ZmlsZXMsIHlvdSd2ZSB0aGUgbGVhc3QgYW1vdW50IG9mIG92ZXJoZWFkIGJ5IGRvaW5nIHNvLg0K DQovSw0KDQo+DQo+ID4NCj4gPiBJIHJhbiBpbnRvIHRoZSBzYW1lIHByb2JsZW0sIHRoZSBzb2x1 dGlvbiB3YXMgdG8gY3JlYXRlIHRoZSB6dm9sIHdpdGggYQ0KPiA+IGxhcmdlciB2b2xibG9ja3Np emUgbGlrZSAzMmsuIElmIHNwYWNlIGVmZmljaWVuY3kgaXMgdGhlIHByaW1hcnkgY29uY2Vybg0K PiA+IEkgYmVsaWV2ZSB5b3UgbmVlZCB0byBjaG9vc2UgYSBibG9ja3NpemUgdGhhdCBpcyBncmVh dGVyIHRoYW4gb3IgZXF1YWwNCj4gPiB0byB0aGUgbnVtYmVyIG9mIG5vbiBwYXJpdHkgZHJpdmVz IG11bHRpcGxpZWQgYnkgdGhlaXIgc2VjdG9yIHNpemUuDQo+ID4gSnVkZ2luZyBieSB5b3VyIG92 ZXJoZWFkIEknbSBnb2luZyB0byBhc3N1bWUgeW91IGhhdmUgYSA2IGRpc2sgcmFpZHoyLA0KPiA+ IHNvIDQgKiA0ayA9IDE2ayBhdCBsZWFzdC4gVHJ5IGNyZWF0aW5nIGFub3RoZXIgenZvbCB3aXRo ICItbw0KPiA+IHZvbGJsb2Nrc2l6ZT0xNmsiIGFuZCBzZWUgaG93IHRoYXQgdHJlYXRzIHlvdS4N Cj4NCj4gUmlnaHQgZ3Vlc3MsDQo+DQo+IEFjdHVhbGx5IHRoZSBpdCBpcyBhIDIqIDYgdm9sIHJh aWR6Mg0KPg0KPiBUSGUgYXJ0aWNsZSBzZWVtcyB0byBzaG93IHRoYXQgdGhlIG92ZXJoZWFkIGdl dHMgbGVzcyBhbmQgbGVzcyB3aGVuIHRoZQ0KPiBibG9ja3NpemUgaW5jcmVhc2VzLg0KPiBJbiB3 aGljaCBjYXNlIEkgc3RhcnQgd29ycnlpbmcgYWJvdXQgdGhlIHBlcmZvcm1hbmNlLCBiZWNhdXNl IHdyaXR0aW5nDQo+IHNtYWxsZXIgYmxvY2tzIHdpbGwgcmVxdWlyZSBhIHJlYWQvd2l0ZSBjeWNs ZS4NCj4NCj4gSW4gbXkgc3BlY2lmaWMgY2FzZSBJJ20gd3JpdGluZyBsYXJnZSB2aWRlbyBmaWxl cyAoNUdiKSBpbiBzdHJlYW1pbmcNCj4gbW9kZSwgc28gdGhlIHJlYWQvd3JpdGUgZWZmZWN0IHdp bGwgYmUgbWl0aWdhdGVkIGR1ZSB0byBzdHJlYW1pbmcuDQo+DQo+IEknbGwgZ28gYW5kIG1ha2Ug YSBmZXcgWlZPTFMgYW5kIHNlZSB3aGF0IGhhcHBlbnMgd2l0aCB0aGUgb3ZlcmhlYWQgd2hlbg0K PiB0aGV5IHN0YXJ0IGZpbGxpbmcgdXAuIEJ1dCBpdCBsb29rcyBsaWtlIHRoZSB3aW5kb3dzIGJs b2Nrc2l6ZSBzaG91bGQNCj4gY2VydGFpbmx5IG5vdCBtYXRjaCB6dm9sIGJsb2Nrc2l6ZS4NCj4N Cj4gPiBBcyB0byB5b3VyIHNlY29uZCBxdWVzdGlvbiwgbW9kZXJuIHZlcnNpb25zIG9mIHdpbmRv d3Mgc2hvdWxkIGFsaWduDQo+ID4gcGFydGl0aW9ucyBjb3JyZWN0bHksIHdoYXQgdmVyc2lvbiBh cmUgeW91IHVzaW5nPyBZb3UgbWF5IHdpc2ggdG8gb3Blbg0KPiA+IGFuIGVsZXZhdGVkIGNvbW1h bmQgcHJvbXB0IGFuZCBzYW5pdHkgY2hlY2sgdGhlIG91dHB1dCBvZiAiZnN1dGlsIGZzaW5mbw0K PiA+IG50ZnNpbmZvIHg6Ig0KPg0KPiBJJ20ganVzdCBsb2FkaW5nIHRoZSBkaXNrIGluIGV4cGxv cmVyLCByaWdodGNsaWNrIGl0LCBhbmQgdGhlbiBmb3JtYXQuDQo+IEp1c3QgYXMgYmFzaWMgYXMg dGhhdC4NCj4gTm90IHN1cmUgd2hhdCB0byBtYWtlIG9mIGl0LCBidXQgdGhpcyBnaXZlczoNCj4N Cj4gQzpcV0lORE9XU1xzeXN0ZW0zMj5mc3V0aWwgZnNpbmZvIG50ZnNpbmZvIG06DQo+IE5URlMg Vm9sdW1lIFNlcmlhbCBOdW1iZXIgOiAgICAgICAgMHgyYWFhNTBiOGFhNTA4MjZkDQo+IE5URlMg VmVyc2lvbiAgIDogICAgICAgICAgICAgICAgICAgMy4xDQo+IExGUyBWZXJzaW9uICAgIDogICAg ICAgICAgICAgICAgICAgMi4wDQo+IE51bWJlciBTZWN0b3JzIDogICAgICAgICAgICAgICAgICAg MHgwMDAwMDAwMjdmZmJlZmZmDQo+IFRvdGFsIENsdXN0ZXJzIDogICAgICAgICAgICAgICAgICAg MHgwMDAwMDAwMDI3ZmZiZWZmDQo+IEZyZWUgQ2x1c3RlcnMgIDogICAgICAgICAgICAgICAgICAg MHgwMDAwMDAwMDAwODQ5N2ZjDQo+IFRvdGFsIFJlc2VydmVkIDogICAgICAgICAgICAgICAgICAg MHgwMDAwMDAwMDAwMGE4ZWMwDQo+IEJ5dGVzIFBlciBTZWN0b3IgIDogICAgICAgICAgICAgICAg NTEyDQo+IEJ5dGVzIFBlciBQaHlzaWNhbCBTZWN0b3IgOiAgICAgICAgNDA5Ng0KPiBCeXRlcyBQ ZXIgQ2x1c3RlciA6ICAgICAgICAgICAgICAgIDgxOTINCj4gQnl0ZXMgUGVyIEZpbGVSZWNvcmQg U2VnbWVudCAgICA6ICAxMDI0DQo+IENsdXN0ZXJzIFBlciBGaWxlUmVjb3JkIFNlZ21lbnQgOiAg MA0KPiBNZnQgVmFsaWQgRGF0YSBMZW5ndGggOiAgICAgICAgICAgIDB4MDAwMDAwMDAwMDE0MDAw MA0KPiBNZnQgU3RhcnQgTGNuICA6ICAgICAgICAgICAgICAgICAgIDB4MDAwMDAwMDAwMDA2MDAw MA0KPiBNZnQyIFN0YXJ0IExjbiA6ICAgICAgICAgICAgICAgICAgIDB4MDAwMDAwMDAwMDAwMDAw MQ0KPiBNZnQgWm9uZSBTdGFydCA6ICAgICAgICAgICAgICAgICAgIDB4MDAwMDAwMDAwMDA2MDBh MA0KPiBNZnQgWm9uZSBFbmQgICA6ICAgICAgICAgICAgICAgICAgIDB4MDAwMDAwMDAwMDA2NjQy MA0KPiBNYXggRGV2aWNlIFRyaW0gRXh0ZW50IENvdW50IDogICAgIDQyOTQ5NjcyOTUNCj4gTWF4 IERldmljZSBUcmltIEJ5dGUgQ291bnQgOiAgICAgICAweGZmZmZmZmZmDQo+IE1heCBWb2x1bWUg VHJpbSBFeHRlbnQgQ291bnQgOiAgICAgNjINCj4gTWF4IFZvbHVtZSBUcmltIEJ5dGUgQ291bnQg OiAgICAgICAweDQwMDAwMDAwDQo+IFJlc291cmNlIE1hbmFnZXIgSWRlbnRpZmllciA6ICAgICBG RDgzNkUxRi0xMzcyLTExRTUtODI1Ri0wMDAyNzIxMzFGM0MNCj4NCj4gVGhlIGdwYXJ0IGluZm8g aW4gdGhlIHp2b2wgaXMgcmVwb3J0ZWQgYXMgdW5hbGlnbmVkIHdoZW4gYm9vdGluZw0KPiBGcmVl QlNELCBidXQgdGhhdCB3b3VsZCBiZSBiZWNhdXNlIHdpbmRvd3MgcHV0IGl0IGluIHRoZXJlIGlu IGFuDQo+IHVuYWxpZ25lZCBwb3N0aXRpb24uIFNvbWV0aGluZyBsaWtlIGF0IDYzICogNTEyIGJ5 dGVzLi4uPz8NCj4NCj4gQW55IGNvbW1hbmRsaW5lIGluY2FybmF0aW9uIHRoYXQgYWxsb3dzIG1l IHRvIHB1dCBpdCBhIGEgZGlmZmVyZW50IG9mZnNldD8NCj4NCj4gPg0KPg0KPiA+IEZpbmFsbHks IGZyb20gbXkgb3duIGV4cGVyaWVuY2Ugd2l0aCB1c2luZyBpc2NzaSB1bmRlciB3aW5kb3dzIEkN Cj4gPiBmb3VuZCBJIGhhZCBhIHJhbmRvbSA0ayBpb3BzIGluY3JlYXNlIGZyb20gfjEwLDAwMCB0 byB+OTAsMDAwICh3aGljaA0KPiA+IGlzIGNsb3NlIHRvIG15IHBvb2xzIHJlYWwgc3BlZWQpIGJ5 IGFkanVzdGluZyB0aGUgZm9sbG93aW5nIHJlZ2lzdHJ5DQo+ID4ga2V5DQo+DQo+DQo+ID4gQ3Jl YXRlIHRoaXMgRFdPUkQga2V5IHdpdGggYSB2YWx1ZSBvZiAxOg0KPiA+IEhLTE1cU1lTVEVNXEN1 cnJlbnRDb250cm9sU2V0XENvbnRyb2xcQ2xhc3NcezREMzZFOTdCLUUzMjUtMTFDRS1CRkMxLTA4 MDAyQkUxMDMxOH1cPEluc3RhbmNlDQo+ID4gTnVtYmVyPlxQYXJhbWV0ZXJzXGlTQ1NJRGlzYWJs ZU5hZ2xlDQo+DQo+IFZlcnkgdXNlZnVsbCBoaW50LCB0aGFueCBmb3IgdGhhdC4NCj4NCj4gLS1X alcNCj4NCj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18N Cj4gZnJlZWJzZC1mc0BmcmVlYnNkLm9yZyBtYWlsaW5nIGxpc3QNCj4gaHR0cHM6Ly9saXN0cy5m cmVlYnNkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2ZyZWVic2QtZnMNCj4gVG8gdW5zdWJzY3JpYmUs IHNlbmQgYW55IG1haWwgdG8gImZyZWVic2QtZnMtdW5zdWJzY3JpYmVAZnJlZWJzZC5vcmciDQo=