From owner-freebsd-fs@freebsd.org Fri Sep 25 14:05:50 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77746A0879B for ; Fri, 25 Sep 2015 14:05:50 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: from mail-ig0-x22d.google.com (mail-ig0-x22d.google.com [IPv6:2607:f8b0:4001:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 421F81F71 for ; Fri, 25 Sep 2015 14:05:50 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: by igcpb10 with SMTP id pb10so12235404igc.1 for ; Fri, 25 Sep 2015 07:05:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=csL/VHuld80mfxArvyDd3XIE/4eC8DSHbWLPgVF/Hyo=; b=Pa3BaHd45RxDsrNkEwKPDMOD+kpWcBTLiv+HePKeGAItgJjQcd6R4ZTIazdhyT1AOs yE7ZbOf5UPfL7nBd86w2lKE/oZYDldR9i1ihhj/JQTroT6nIDzvYdeVcNLE8k9ySi8Em fwhQkvLtPOK/aJSGlaSHjj6GXAH4pfA2+svqMk+s0UYNRjM47XanhhlFrYP90Jp03Ukz L+YEuGVutDPtX3bxBuwbmCga6QVrm75K18PyqkLji9SVf08vJtJ+jKhtLMHaEPHDO051 CnJOJN8JUhXVi6Aa+Cl34j/kvcMi9eihZPcdUQHpDTIao56jU6dkbBIV561lZZbTBpjT avTg== MIME-Version: 1.0 X-Received: by 10.50.134.161 with SMTP id pl1mr2928975igb.28.1443189949525; Fri, 25 Sep 2015 07:05:49 -0700 (PDT) Received: by 10.107.4.72 with HTTP; Fri, 25 Sep 2015 07:05:49 -0700 (PDT) In-Reply-To: <1443179614.5271.42.camel@data-b104.adm.slu.se> References: <56050EFC.1000303@digiware.nl> <1443179614.5271.42.camel@data-b104.adm.slu.se> Date: Fri, 25 Sep 2015 10:05:49 -0400 Message-ID: Subject: Re: ZFS, Zvol, iSCSI and windows From: Tom Curry To: Willem Jan Withagen Cc: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Sep 2015 14:05:50 -0000 On Fri, Sep 25, 2015 at 7:13 AM, Karli Sj=C3=B6berg = wrote: > fre 2015-09-25 klockan 11:08 +0200 skrev Willem Jan Withagen: > > Hi, > > > > Because of the Network Video Recorder (on windows) we use only likes > > "real" disks, and not SMB disks. We started using ZVOLs which are > > exported thru iSCSI/ctld.... > > > > And that works really well, so there all thumbs up for this combo. > > > > However..... (you knew that was coming) > > I do have some questions, and hope somebody can share some insights. > > > > 1) > > This is a ZFS question. > > > > I have created a ZVOL with: > > zfs create -s -V 5T zfsraid/nvr2 > > > > Looking at the disk usage in 'zfs get all': > > NAME PROPERTY VALUE SOURCE > > zfsraid/nvr2 used 9.97T - > > zfsraid/nvr2 available 438G - > > zfsraid/nvr2 referenced 9.97T - > > zfsraid/nvr2 compressratio 1.00x - > > zfsraid/nvr2 reservation none default > > zfsraid/nvr2 volsize 5T local > > zfsraid/nvr2 volblocksize 8K - > > zfsraid/nvr2 checksum on default > > zfsraid/nvr2 compression lz4 default > > zfsraid/nvr2 primarycache all default > > zfsraid/nvr2 secondarycache all default > > zfsraid/nvr2 usedbysnapshots 0 - > > zfsraid/nvr2 usedbydataset 9.97T - > > zfsraid/nvr2 usedbychildren 0 - > > zfsraid/nvr2 usedbyrefreservation 0 - > > zfsraid/nvr2 sync standard default > > zfsraid/nvr2 written 9.97T - > > zfsraid/nvr2 logicalused 4.97T - > > zfsraid/nvr2 logicalreferenced 4.97T - > > zfsraid/nvr2 volmode default default > > > > And what sort of "worries" me is that it seems that this volume is usin= g > > twice the amount of diskspace it is offering as ZVOL? > > > > a) Is this really true? > > b) Should I have done something different to not waste so much overhead= ? > > > > Note that the compression rate is 1.00x, which is of course due to > > writing h264 media streams that do not compress at all. But is the > > parents default, and I haven't turned it off. > > > > 2) > > The second one might be more a Windows question, but anyways. > > > > The export is that same 5T ZVOL, plain create with > > zfs create -s -V 5T zfsraid/nvr2 > > Under Windows I used the regular stuff to format the dis with GPT and > > NTFS with 8k segments. (matching the ZVOL blocksize) > > > > Upon reboot FreeBSD notices the following: > > GEOM_PART: partition 1 on (zvol/zfsraid/nvr2, GPT) is not aligned on > > 8192 bytes > > By itself not technical problem. But the warning does hint that the > > alignment is thus, that performance might suffer. the backing disks are > > WD REDs which are 4K sectors.... So the misalignment could cause too > > many read/writes for writing. > > > > Does anybody have a clue as how to get Windows to do the aligning > correctly? > > > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > Is it this effect you are seeing: > > https://forums.freebsd.org/threads/mitigating-4k-disk-issues-on-raidz.373= 65/ > > /K > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > I ran into the same problem, the solution was to create the zvol with a larger volblocksize like 32k. If space efficiency is the primary concern I believe you need to choose a blocksize that is greater than or equal to the number of non parity drives multiplied by their sector size. Judging by your overhead I'm going to assume you have a 6 disk raidz2, so 4 * 4k =3D 1= 6k at least. Try creating another zvol with "-o volblocksize=3D16k" and see ho= w that treats you. As to your second question, modern versions of windows should align partitions correctly, what version are you using? You may wish to open an elevated command prompt and sanity check the output of "fsutil fsinfo ntfsinfo x:" Finally, from my own experience with using iscsi under windows I found I had a random 4k iops increase from ~10,000 to ~90,000 (which is close to my pools real speed) by adjusting the following registry key Create this DWORD key with a value of 1: HKLM\SYSTEM\CurrentControlSet\Control\Class\{4D36E97B-E325-11CE-BFC1-08002B= E10318}\\Parameters\iSCSIDisableNagle