From owner-freebsd-fs@freebsd.org Sun Jan 15 09:54:39 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7353ACA4B80 for ; Sun, 15 Jan 2017 09:54:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 62ED717EB for ; Sun, 15 Jan 2017 09:54:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0F9scnp013243 for ; Sun, 15 Jan 2017 09:54:39 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 182739] [fusefs] [panic] sysutils/fusefs-kmod kernel panic on rsync Date: Sun, 15 Jan 2017 09:54:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rene@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 09:54:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D182739 Rene Ladan changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rene@FreeBSD.org Resolution|--- |FIXED Status|In Progress |Closed --- Comment #3 from Rene Ladan --- Close this PR, this port was only required on FreeBSD < 10.0 which are now = all end-of-life. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 15 21:01:02 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF09BCB137E for ; Sun, 15 Jan 2017 21:01:02 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6D15014DE for ; Sun, 15 Jan 2017 21:01:02 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0FL01JC027183 for ; Sun, 15 Jan 2017 21:01:02 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201701152101.v0FL01JC027183@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention Date: Sun, 15 Jan 2017 21:01:02 +0000 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 21:01:03 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 140068 | [smbfs] [patch] smbfs does not allow semicolon in Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 203419 | solaris assert: (dn->dn_phys->dn_nlevels == 0 && Open | 211491 | System hangs after "Uptime" on reboot with ZFS 7 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Mon Jan 16 17:59:44 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9BD8FCB249A; Mon, 16 Jan 2017 17:59:44 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: from mail-yw0-x233.google.com (mail-yw0-x233.google.com [IPv6:2607:f8b0:4002:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E4E71F8D; Mon, 16 Jan 2017 17:59:44 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: by mail-yw0-x233.google.com with SMTP id a10so73068394ywa.3; Mon, 16 Jan 2017 09:59:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=uBqM3tOjV42aw/X1gizj8bf4UI6tpvuysLLgPJfAVvY=; b=jnNOsk0gXAHPcklcnbPaqfrCf8a+KsxFS3NjL1vmhMdtRUTZSDapg12ClDuwu6VmAc mdAJD3jYX/uBpkHyslQJ7HBooBDRUJYfgZlciF+S1OwycA7O8yZMDfEbRvyzTsPEKiE9 k+h5xtM5oO1/TkcYHRiN9OXIJD8aT/CxNziO5LnNwHGMd5Oc2pGE8+7bDgZ7Ai8age2O x3R1LLxFwms3eYF9krZpjh5dPVQ1lJnrVxkgYSUulG//lyvOcPNVlN2TzzN5sl7T0A3r 3k3FGCwya+7F7Tuy4Q64NounJV5m4Ed8nZZlwpKoQntAaBcFpB5LsGjzAZY8d7+oBdYl M9Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=uBqM3tOjV42aw/X1gizj8bf4UI6tpvuysLLgPJfAVvY=; b=Y/Vo6rXrx6Du0nhED777/OHr3NCmMZYoHQ8If0rdJHEaAU5RAV2mFis65nS2t7AtX5 GcwcFQ9ibTrD1g1JmsZjrsHu6GSawGmoRxu4/fmI2nSvnVrECChIN4lyALpnb+hXWsMY LL2stTCLn70NPTaIwv8btWqY/syW35/6jauqZNWK9PHFV7Qiws2J/Yf0GJy4/0QACePa NMHAaP/8B9pJdiKGBBuaMtPHot1fvihaTEawBIACqI46BpX2jFV9Dnx9Dukcuzvdr20a rnqG9IjmG72VMxOXFy6jIPxo1cU45nSUJKIhX5Mt6hPjt8FJsbwLllqVeePr0qhue5MU tOCQ== X-Gm-Message-State: AIkVDXKX5Rxgv3u+9asBfDNF/WZ5Vx31jeHZUOAHkaOBxqVjD3r47rcKpLbGc8u8U3ik91HE+McKJ0xdph125Q== X-Received: by 10.129.122.200 with SMTP id v191mr6239364ywc.114.1484589583402; Mon, 16 Jan 2017 09:59:43 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.52.65 with HTTP; Mon, 16 Jan 2017 09:59:43 -0800 (PST) From: Ultima Date: Mon, 16 Jan 2017 12:59:43 -0500 Message-ID: Subject: zfs zvol's inaccessible after reboot To: freebsd-current@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jan 2017 17:59:44 -0000 Currently there is a bug with zvols. I have a few Bhyve containers that startup at boot. I'v noticed in middle December of last year that after a restart the zvols become inaccessible to the container. Nothing can be done to the zvol, other than rename. It cannot even be destroyed in this state. The only way to make it accessible again is to renaming the zvol, after this occurs, functionality is restored. The bug is still present in head r312232. From owner-freebsd-fs@freebsd.org Tue Jan 17 15:11:13 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C766CCB391C for ; Tue, 17 Jan 2017 15:11:13 +0000 (UTC) (envelope-from gmodserver@vulture-gaming.de) Received: from vulture-gaming.de (vulture-gaming.de [89.163.225.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 78BE519F1 for ; Tue, 17 Jan 2017 15:11:12 +0000 (UTC) (envelope-from gmodserver@vulture-gaming.de) Received: by vulture-gaming.de (Postfix, from userid 1000) id 347155E21DF; Tue, 17 Jan 2017 16:04:41 +0100 (CET) To: freebsd-fs@freebsd.org Subject: Your Apple account information has been changed Message-ID: <1484665481.18977.qmail@service.Pay-pal.co.uk> From: "Apple" Date: Tue, 17 Jan 2017 16:04:41 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2017 15:11:13 -0000 From owner-freebsd-fs@freebsd.org Tue Jan 17 19:28:08 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32A66CB4373 for ; Tue, 17 Jan 2017 19:28:08 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E6FD81D9E for ; Tue, 17 Jan 2017 19:28:07 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1cTZQK-0003yt-TF; Tue, 17 Jan 2017 20:28:05 +0100 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: "Michael Ware" , "Malte Graebner" Cc: "freebsd-fs@freebsd.org" Subject: Re: NFS 4.1 exports References: Date: Tue, 17 Jan 2017 20:28:03 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.2 X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 autolearn=disabled version=3.4.0 X-Scan-Signature: 7006e789400ccdeb5065f5d065827fb5 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2017 19:28:08 -0000 Hello, 'man nfsv4' gives some information (on 12-STABLE). I don't know if it is 4.0 or 4.1 or what the exact difference is. I don't run it myself currently. Regards, Ronald. On Fri, 13 Jan 2017 19:51:51 +0100, Malte Graebner wrote: > Hi Micheal, > > are you looking for this ? > > https://www.freebsd.org/cgi/man.cgi?query=nfsv4&sektion=4 > > # man nfsd > > As described in the man page nfsd is using nfs4 isn't it ? > > > br, > > malte > > Am 01/13/17 um 19:42 schrieb Michael Ware: >> Hi Malte, >> Thanks for the reply. I have seen this previously although I am running >> a standard nfs daemon at this point. There doesn't seem to be a way to >> set the version of export through. >> Mike >> >> On Fri, Jan 13, 2017 at 10:21 AM, Malte Graebner > > wrote: >> >> Take a look at zfs share / zfs export >> >> https://docs.oracle.com/cd/E23824_01/html/821-1448/gayne.html >> >> >> br, >> malte >> >> Am 01/13/17 um 18:49 schrieb Michael Ware: >> >> I just installed freebsd 11 and am looking to set up an NFS >> 4.1 server on >> it. I have not been able to find any documentation on this. Is >> this feature >> not available yet? >> >> Thanks >> Mike >> >> >> >> >> -- Michael Ware >> UCSC Baskin Engineering >> Unix, Network and Security >> 406-210-4725 > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Wed Jan 18 06:00:20 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2815ECB38B4 for ; Wed, 18 Jan 2017 06:00:20 +0000 (UTC) (envelope-from hiyorin@gmail.com) Received: from mail-lf0-x231.google.com (mail-lf0-x231.google.com [IPv6:2a00:1450:4010:c07::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A1C401394 for ; Wed, 18 Jan 2017 06:00:19 +0000 (UTC) (envelope-from hiyorin@gmail.com) Received: by mail-lf0-x231.google.com with SMTP id z134so2124720lff.3 for ; Tue, 17 Jan 2017 22:00:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=N5iukBVaEVWQSFD3PDeFx6HWRcW/rVnCN2unDv7JTLs=; b=mwkRnnP7KU1p2U+Fu0YJz7ruYxsUVtKCjgzczC18N3scXTUCATQNPYp1VaHALEupWp PAeRKEdapYt5K6VlBZqBR/j8NWl9cGMiqDjKS8ZbfQwmMCMwM0lCp3t2RlBHaOHRe/cy Dz0zs1D2a7pwyVPNmWI+H6QGnbaWNGAO3xnCGjN5dbyfv2KNJUCgvqqIRFX801ZUF3ZR oB4QEtxxGuVPy/FSr4TuH7hvSw/Fj7OQPVqtxA965f6GAtdkmmoTfitdQFNLNjG7fjY3 l3GF4nREf+Wrl8ebqdH+RoJo0TEUoPMUTzWf2YgLlkPISYNmb4uPsZ3ymlXmLPfLlIIE roQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=N5iukBVaEVWQSFD3PDeFx6HWRcW/rVnCN2unDv7JTLs=; b=APUzj4lChufEH4QSRbcd1LxH1qj69yt4zl4X/R+6VP6wST0m9uI5kj4mMSk5y1gY1F 1amypdWC6m1zimWpDMKtJw0uy8GsKsDxbScoo9YdrHYK5EIp8vlbGwHqL34Vv7eVjpDF USAq2/xMTc7z4p+tShz3SGRGyjoQKlHD18zpcCSE3c41NoFZimPh0/wLqIQt2w7B2fBZ qJeD3rNUeBV8cAwQhJ2e5I0e/thSdarj7U6Sd6C1VTErOl/sTdOnTqBLveV6MXiNoVku tfI3BZHmxMht1RY/thnB+tR8yuctH7zyGRfFJ4spFcrvze1JFG6NPvbHxMoyAj90BLOh McTg== X-Gm-Message-State: AIkVDXJKh5VMXXwwVP0cszFbke2v48wUE89/FQlbdWPJrgDF7xTQrduqKmYz5uVL1glyUvZHs0fC9JuNVypdUA== X-Received: by 10.25.205.77 with SMTP id d74mr457992lfg.122.1484719217697; Tue, 17 Jan 2017 22:00:17 -0800 (PST) MIME-Version: 1.0 From: "C. C. Tang" Date: Wed, 18 Jan 2017 06:00:07 +0000 Message-ID: Subject: [fusefs] default_permission + allow_other not work as expected. To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 06:00:20 -0000 uname -a FreeBSD bsd_test 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 Problem: according to manual of mount.fuse, mounting fusefs with option default_permission and allow_other should allow access of mounted data by other user and enable standard file permission checking by the system. However it seems the permission check is not working as expected on FreeBSD. I haved tried to reproduce it on CentOS7 and it works as expected. (user1 get permission denied error) How to reproduce: (use unionfs as example) as root: # cd /_test # mkdir a # mkdir b # echo "1" > a/1 # echo "2" > a/2 # chmod 600 a/1 # mkdir u # unionfs -o default_permissions,allow_other a=RW:b=RO u # ll u total 8 -rw------- 1 root wheel 2 Jan 18 21:23 1 -rw-r--r-- 1 root wheel 2 Jan 18 21:23 2 # cat a/1 1 ============== as a normal user % cd /_test % cat a/1 cat: a/1: Permission denied % cat u/1 1 Expected output: as a normal user % cd /_test % cat a/1 cat: a/1: Permission denied % cat u/1 cat: u/1: Permission denied From owner-freebsd-fs@freebsd.org Wed Jan 18 09:35:11 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5F38CB5FFA for ; Wed, 18 Jan 2017 09:35:11 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C9591604 for ; Wed, 18 Jan 2017 09:35:11 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1cTme4-00080x-N4; Wed, 18 Jan 2017 10:35:09 +0100 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: "Rick Macklem" Subject: Re: NFS 4.1 exports References: Date: Wed, 18 Jan 2017 10:35:07 +0100 Cc: "freebsd-fs@freebsd.org" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.2 X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 autolearn=disabled version=3.4.0 X-Scan-Signature: a9e4b997d6a751f3e45cb47a3c2b1d2c X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 09:35:11 -0000 The original question was not posted by me, so I cc this answer back into the mailinglist as Rick is quite knowledgeable in the NFS stuff of FreeBSD. Ronald. On Tue, 17 Jan 2017 23:35:38 +0100, Rick Macklem wrote: > The server supports both NFSv4.0 and NFSv4.1 and the exports are the > same for both. > `man exports` is confusing, but you need the same lines as you would > have to export > the file systems for NFSv3 + a line that defines where the root of the > NFSv4 directory tree is. > This additional line does not export any file systems, so you still need > all the lines for the > file systems. This additional line starts with ``V4: ...``. > (FreeBSD9 only supported NFSv4.0, but 10,11,current support 4.1 as well.) > > rick > > ________________________________________ > From: owner-freebsd-fs@freebsd.org on > behalf of Ronald Klop > Sent: Tuesday, January 17, 2017 2:28:03 PM > To: Michael Ware; Malte Graebner > Cc: freebsd-fs@freebsd.org > Subject: Re: NFS 4.1 exports > > Hello, > > 'man nfsv4' gives some information (on 12-STABLE). I don't know if it is > 4.0 or 4.1 or what the exact difference is. I don't run it myself > currently. > > Regards, > Ronald. > > > On Fri, 13 Jan 2017 19:51:51 +0100, Malte Graebner > wrote: > >> Hi Micheal, >> >> are you looking for this ? >> >> https://www.freebsd.org/cgi/man.cgi?query=nfsv4&sektion=4 >> >> # man nfsd >> >> As described in the man page nfsd is using nfs4 isn't it ? >> >> >> br, >> >> malte >> >> Am 01/13/17 um 19:42 schrieb Michael Ware: >>> Hi Malte, >>> Thanks for the reply. I have seen this previously although I am running >>> a standard nfs daemon at this point. There doesn't seem to be a way to >>> set the version of export through. >>> Mike >>> >>> On Fri, Jan 13, 2017 at 10:21 AM, Malte Graebner >> > wrote: >>> >>> Take a look at zfs share / zfs export >>> >>> https://docs.oracle.com/cd/E23824_01/html/821-1448/gayne.html >>> >>> >>> br, >>> malte >>> >>> Am 01/13/17 um 18:49 schrieb Michael Ware: >>> >>> I just installed freebsd 11 and am looking to set up an NFS >>> 4.1 server on >>> it. I have not been able to find any documentation on this. Is >>> this feature >>> not available yet? >>> >>> Thanks >>> Mike >>> >>> >>> >>> >>> -- Michael Ware >>> UCSC Baskin Engineering >>> Unix, Network and Security >>> 406-210-4725 >> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Wed Jan 18 10:04:53 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24B38CB53A5 for ; Wed, 18 Jan 2017 10:04:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 146881EE1 for ; Wed, 18 Jan 2017 10:04:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0IA4AwF057286 for ; Wed, 18 Jan 2017 10:04:52 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 207464] Panic when destroying ZFS snapshot Date: Wed, 18 Jan 2017 10:04:10 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: daniel.genis@gmx.de X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: avg@FreeBSD.org X-Bugzilla-Flags: mfc-stable9+ mfc-stable10+ X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 10:04:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D207464 Daniel G changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |daniel.genis@gmx.de --- Comment #32 from Daniel G --- I would also love to see this in the 10.3 releng branch --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Wed Jan 18 16:27:49 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4072ECB6C10 for ; Wed, 18 Jan 2017 16:27:49 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) by mx1.freebsd.org (Postfix) with ESMTP id 09BDD16A0; Wed, 18 Jan 2017 16:27:49 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [192.168.17.133] (unknown [89.113.128.32]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id C8C46A63; Wed, 18 Jan 2017 19:27:47 +0300 (MSK) Reply-To: lev@FreeBSD.org To: freebsd-fs@freebsd.org Cc: Andriy Gapon , Alexander Motin From: Lev Serebryakov Subject: Could somebody look at PR216178? Organization: FreeBSD Message-ID: <2cadd04c-7b7c-9963-489d-d211215a1d3f@FreeBSD.org> Date: Wed, 18 Jan 2017 19:27:42 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="AbARR3OeCGu09pjsJAh2HIR5QmEN875Cq" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 16:27:49 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --AbARR3OeCGu09pjsJAh2HIR5QmEN875Cq Content-Type: multipart/mixed; boundary="7WjpHJ8auVJR0kSNGfswgjs2tdbtS6Drp"; protected-headers="v1" From: Lev Serebryakov Reply-To: lev@FreeBSD.org To: freebsd-fs@freebsd.org Cc: Andriy Gapon , Alexander Motin Message-ID: <2cadd04c-7b7c-9963-489d-d211215a1d3f@FreeBSD.org> Subject: Could somebody look at PR216178? --7WjpHJ8auVJR0kSNGfswgjs2tdbtS6Drp Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 I could reproduce it on 10-STABLE and 11-STABLE and I'm afraid, that this ruin ARC and L2ARC efficiency completely. --=20 // Lev Serebryakov --7WjpHJ8auVJR0kSNGfswgjs2tdbtS6Drp-- --AbARR3OeCGu09pjsJAh2HIR5QmEN875Cq Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQJ8BAEBCgBmBQJYf5d+XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGOTZEMUNBMEI1RjQzMThCNjc0QjMzMEFF QUIwM0M1OEJGREM0NzhGAAoJEOqwPFi/3EePQ1MQAKagZMjk+1pRK6nz/yR6Ooc5 4A7+rk0/mZ8coadHAivhSLAykhVRSV6V0UtAidrgSZg5GKR+UWTqEXjxacUee8W0 aJS6bIEpcxhSdDCD7YnbiFdg80g31bLr3fl2unL3VPBWWfTNU+NaM54a31FB1DGy IQgjbWzKeTwcGDimpcZpZ8+4AG3XJ1hyqNbUGVNswzQcY48JGBWuN+2JH6zFpVoY 3dxaL8A4xLKkulJ2wEr6aal29aMehAjfvF+ncJhPpMJMePEvtKxZ5gPLMZ/pQ+wp uzpTZo/3gyMbyWX8B9aC6qdQYQt5lgdymKPQuV95RIKPSf4E9Uko3Xdoldn8kn91 xvt3X7XprHiHYfApBhjDJwoHbj2k2PCogqFkrrLl7l0VLZfvJG4po39RRk3A6zA4 HTjd7s0vOBd4bZr/Jx4Wpbj8bNOyBSbKur5jQ8+TWIbQo8MVFKmoNxmUwEA3wENu R7IfoCzGJCR4fT7Ux96ZR+RwEru8pPDOGIcklf30B5zkPfBkHHoJLHfEEyqqQB3p MfI7ennQgoqZQ+dtDFZoAFXHfrc7/Im5q/cs313dQZYutPDj8M468ZBUdMFMdRZB LKFlA8evq5fMxWUAUsbf/I2LuaO5U95NCO7pOXW1KxsqsSdDUZXhIKpnynJFV2fk SLjy1JOMSy5NG7djufZS =U1wx -----END PGP SIGNATURE----- --AbARR3OeCGu09pjsJAh2HIR5QmEN875Cq-- From owner-freebsd-fs@freebsd.org Wed Jan 18 16:44:24 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 255ADCB60D9 for ; Wed, 18 Jan 2017 16:44:24 +0000 (UTC) (envelope-from maware@ucsc.edu) Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DD66E128C for ; Wed, 18 Jan 2017 16:44:23 +0000 (UTC) (envelope-from maware@ucsc.edu) Received: by mail-oi0-x231.google.com with SMTP id u143so9897356oif.3 for ; Wed, 18 Jan 2017 08:44:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=P4sEHea5tmmira0JtsUcNLVsTl1BLLI19i661JnE2Lo=; b=bP3zJmEeuGEroCdc9GHGTPBqaSOtwRE0vOXvFJS0/PFPVS260wqp96qiWl9ZltoLaD AgcLsAq6vcWFgMXqgjwic3CG9avJaoQPSWQUEM4xGE0CynD3T3Ozy5/XdwsCeVN2S1mW 5Sc6VhiKWrpeQwBYte819IbTY8JEkLpnW8UUw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=P4sEHea5tmmira0JtsUcNLVsTl1BLLI19i661JnE2Lo=; b=S/FY5yJ0+V4JroStiid/dXzjYEJI+kmA8bFQuY+p061y4jsQwxrxl5j3B92UqQZyt6 8H78DntHFOWrHr7auJlLTa36DVshBHcf9K98UoC/vcqHIgRY83oXY1Br80U/teYqYRcz DiILkDj6URZMS0Oiu2lVx+HMOjyad1MLoF4uQKMxzeGRIQkV8Kp3vWyOsG9EEnWqZA+z CN75ifmQhWdGAdB9XvjnzvzlefSdVlQhubU22uEUdVBW0iP1PAPCI1aFek0wLprx9auS XVkXngdyBsu7sLx2xfb7LVl5NeaBoWLEgxq3shayId/SNG0nNkJkBDp/fIN1dvtNTxIn 93mA== X-Gm-Message-State: AIkVDXLDnB2sA81n5vyX4v1hEP+6FQZ6kQpFOwt3GarJYPBdYOKU9ZUDXU/NE1cXMRypludugs9OKzaZgbM4IegN X-Received: by 10.202.175.68 with SMTP id y65mr2241321oie.187.1484757863111; Wed, 18 Jan 2017 08:44:23 -0800 (PST) MIME-Version: 1.0 Received: by 10.74.59.136 with HTTP; Wed, 18 Jan 2017 08:44:22 -0800 (PST) In-Reply-To: References: From: Michael Ware Date: Wed, 18 Jan 2017 08:44:22 -0800 Message-ID: Subject: Re: NFS 4.1 exports To: Ronald Klop Cc: Rick Macklem , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 16:44:24 -0000 Just to give more info: rc.conf I have nfsv4_server_enable="yes" in /etc/exports V4: / -sec=sys -network 172.20.8.0/24 /export/vmimages -maproot=root -network 172.20.8.0/24 Is there a minor revision option for the v4 export line I'm missing? If so 11 has no documentation on it On Wed, Jan 18, 2017 at 1:35 AM, Ronald Klop wrote: > The original question was not posted by me, so I cc this answer back into > the mailinglist as Rick is quite knowledgeable in the NFS stuff of FreeBSD. > > Ronald. > > On Tue, 17 Jan 2017 23:35:38 +0100, Rick Macklem > wrote: > > The server supports both NFSv4.0 and NFSv4.1 and the exports are the same >> for both. >> `man exports` is confusing, but you need the same lines as you would have >> to export >> the file systems for NFSv3 + a line that defines where the root of the >> NFSv4 directory tree is. >> This additional line does not export any file systems, so you still need >> all the lines for the >> file systems. This additional line starts with ``V4: ...``. >> (FreeBSD9 only supported NFSv4.0, but 10,11,current support 4.1 as well.) >> >> rick >> >> ________________________________________ >> From: owner-freebsd-fs@freebsd.org on >> behalf of Ronald Klop >> Sent: Tuesday, January 17, 2017 2:28:03 PM >> To: Michael Ware; Malte Graebner >> Cc: freebsd-fs@freebsd.org >> Subject: Re: NFS 4.1 exports >> >> >> Hello, >> >> 'man nfsv4' gives some information (on 12-STABLE). I don't know if it is >> 4.0 or 4.1 or what the exact difference is. I don't run it myself >> currently. >> >> Regards, >> Ronald. >> >> >> On Fri, 13 Jan 2017 19:51:51 +0100, Malte Graebner >> wrote: >> >> Hi Micheal, >>> >>> are you looking for this ? >>> >>> https://www.freebsd.org/cgi/man.cgi?query=nfsv4&sektion=4 >>> >>> # man nfsd >>> >>> As described in the man page nfsd is using nfs4 isn't it ? >>> >>> >>> br, >>> >>> malte >>> >>> Am 01/13/17 um 19:42 schrieb Michael Ware: >>> >>>> Hi Malte, >>>> Thanks for the reply. I have seen this previously although I am running >>>> a standard nfs daemon at this point. There doesn't seem to be a way to >>>> set the version of export through. >>>> Mike >>>> >>>> On Fri, Jan 13, 2017 at 10:21 AM, Malte Graebner >>> > wrote: >>>> >>>> Take a look at zfs share / zfs export >>>> >>>> https://docs.oracle.com/cd/E23824_01/html/821-1448/gayne.html >>>> >>>> >>>> br, >>>> malte >>>> >>>> Am 01/13/17 um 18:49 schrieb Michael Ware: >>>> >>>> I just installed freebsd 11 and am looking to set up an NFS >>>> 4.1 server on >>>> it. I have not been able to find any documentation on this. Is >>>> this feature >>>> not available yet? >>>> >>>> Thanks >>>> Mike >>>> >>>> >>>> >>>> >>>> -- Michael Ware >>>> UCSC Baskin Engineering >>>> Unix, Network and Security >>>> 406-210-4725 >>>> >>> >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > -- Michael Ware UCSC Baskin Engineering Unix, Network and Security 406-210-4725 From owner-freebsd-fs@freebsd.org Wed Jan 18 17:35:37 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D349CB6530 for ; Wed, 18 Jan 2017 17:35:37 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: from h2.pinyon.org (h2.pinyon.org [65.101.20.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6C74517CA for ; Wed, 18 Jan 2017 17:35:37 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: by h2.pinyon.org (Postfix, from userid 58) id F09811F732; Wed, 18 Jan 2017 10:35:35 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=pinyon.org; s=DKIM; t=1484760935; bh=dFvtC60lc8IhCKVyZgn7+kLdaas28RdU+RvIsJYbXQI=; h=Subject:To:References:From:Date:In-Reply-To; b=SPpn1cqCS4AxZVLRNXfi8YzLaTeTnHdX2DcAA43N6NfulNPW42TDuzkhEYdzt9xBH KlhCwLxQBZk4NZnl09dRbD46Z9pVaAPVPiOpE2+18nqAJRUfxJZoxXnBl3PMneAUBL jwxsrw3zgXL/fKP2xUKanhfNiwkIEArokLa87Z/c= X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on h2.pinyon.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from [10.0.10.15] (h1.pinyon.org [65.101.20.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by h2.pinyon.org (Postfix) with ESMTPSA id 309951F71D for ; Wed, 18 Jan 2017 10:35:35 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=pinyon.org; s=DKIM; t=1484760935; bh=dFvtC60lc8IhCKVyZgn7+kLdaas28RdU+RvIsJYbXQI=; h=Subject:To:References:From:Date:In-Reply-To; b=SPpn1cqCS4AxZVLRNXfi8YzLaTeTnHdX2DcAA43N6NfulNPW42TDuzkhEYdzt9xBH KlhCwLxQBZk4NZnl09dRbD46Z9pVaAPVPiOpE2+18nqAJRUfxJZoxXnBl3PMneAUBL jwxsrw3zgXL/fKP2xUKanhfNiwkIEArokLa87Z/c= Subject: Re: NFS 4.1 exports To: freebsd-fs@freebsd.org References: From: "Russell L. Carter" Message-ID: <23392774-b72a-76c2-b637-6c592b504016@pinyon.org> Date: Wed, 18 Jan 2017 10:35:34 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 17:35:37 -0000 On 01/18/17 09:44, Michael Ware wrote: > Just to give more info: > rc.conf I have nfsv4_server_enable="yes" > > in /etc/exports > V4: / -sec=sys -network 172.20.8.0/24 > /export/vmimages -maproot=root -network 172.20.8.0/24 > This is what I use, no matter what subset of the filesystem is being exported: / -maproot=root V4: / -network 10.0.0.0/16 A surprisingly large amount of words exists in the lists about why it can't be much more complicated than this. Russell From owner-freebsd-fs@freebsd.org Wed Jan 18 19:06:12 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9FA0BCB6460 for ; Wed, 18 Jan 2017 19:06:12 +0000 (UTC) (envelope-from maware@ucsc.edu) Received: from mail-ot0-x231.google.com (mail-ot0-x231.google.com [IPv6:2607:f8b0:4003:c0f::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63A3119BC for ; Wed, 18 Jan 2017 19:06:12 +0000 (UTC) (envelope-from maware@ucsc.edu) Received: by mail-ot0-x231.google.com with SMTP id f9so15579248otd.1 for ; Wed, 18 Jan 2017 11:06:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=EyVYqITOU/e5Jv/THGV8+TYpO7nYcuRIZ+ZkhYdmfTk=; b=e3/fzuLSbYrdFlbm9MftZDqakcSAL+XwZ0KJISYT4jcWdLQOK64LDIL3GebAGsHX3K ceR5abypWDwtKOJschbRR6tSsuXlxH+gne6eZsXZegp/98XxJ7a1u/3eW1LAurSGu7JW b6xST6UfqazQHA1E0DdELt+VvdYA286WclhVw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=EyVYqITOU/e5Jv/THGV8+TYpO7nYcuRIZ+ZkhYdmfTk=; b=rjCTy8/subyLNN81NRHesSx6YVZAnxii952nwlVAlP2Pdb1jEUUGqu8LM0U6byKRaT ToYGbZTgZLfQ/+znEE+lBf0c8Xq01k4HXTjwY0D1eVQps0wkWJhXomHSJ/qdsJL8hpSA ALI7YHnj+PZdC9U69ISvHzg2LdYBfL4w3nXes4gSHlUhIaQPd2JewDvpP6+SlLeKAjuf xzAQwAZUAjkiFA+wCq29pirtD2nwThltqYLS081IWkERaCZD6Z+3o8BZVzh7VyviM7EN PmnFXDfUkrMeseWzC54GrgsvNG6n/ps302EHmOUo3G/1dl7w30SGYF5MxETsoiPEjek9 bMYw== X-Gm-Message-State: AIkVDXJxi2dvWqZXaE3Lb5HFXLGnvDPD4rTQn9jSTHNgrlqakBAIRge3ECDXcFT9a8G6f3iKwE9fi6fl3G/j0jJ1 X-Received: by 10.157.37.209 with SMTP id q75mr2555603ota.267.1484766371618; Wed, 18 Jan 2017 11:06:11 -0800 (PST) MIME-Version: 1.0 Received: by 10.74.59.136 with HTTP; Wed, 18 Jan 2017 11:06:11 -0800 (PST) In-Reply-To: <23392774-b72a-76c2-b637-6c592b504016@pinyon.org> References: <23392774-b72a-76c2-b637-6c592b504016@pinyon.org> From: Michael Ware Date: Wed, 18 Jan 2017 11:06:11 -0800 Message-ID: Subject: Re: NFS 4.1 exports To: "Russell L. Carter" Cc: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 19:06:12 -0000 So just for more info I tried to mount using the esxicli command with the following results. [root@vmserver:~] esxcli storage nfs41 add -H storagearray.private -s /export/vms -v vmstorage Sysinfo error on operation returned status : Timeout. Please see the VMkernel log for detailed error information 2017-01-18T19:00:27.779Z cpu4:2983706 opID=e9567d5d)NFS41: NFS41_VSIMountSet:402: Mount server: storagearray.private, port: 2049, path: /export/vms, label: vmstorage, security: 1 user: , options: 2017-01-18T19:00:27.779Z cpu4:2983706 opID=e9567d5d)StorageApdHandler: 982: APD Handle Created with lock[StorageApd-0x4308ab000130] 2017-01-18T19:00:42.781Z cpu16:4324289)WARNING: SunRPC: 3947: fail all pending calls for client 0x430440b90ee0 (socket disconnected) 2017-01-18T19:00:47.779Z cpu0:2983706 opID=e9567d5d)WARNING: NFS41: NFS41FSWaitForCluster:3433: Failed to wait for the cluster to be located: Timeout 2017-01-18T19:00:47.779Z cpu0:2983706 opID=e9567d5d)WARNING: NFS41: NFS41_FSMount:4412: NFS41FSDoMount failed: Timeout 2017-01-18T19:00:47.779Z cpu0:2983706 opID=e9567d5d)StorageApdHandler: 1066: Freeing APD handle 0x4308ab000130 [] 2017-01-18T19:00:47.779Z cpu0:2983706 opID=e9567d5d)StorageApdHandler: 1150: APD Handle freed! 2017-01-18T19:00:47.779Z cpu0:2983706 opID=e9567d5d)WARNING: NFS41: NFS41_VSIMountSet:410: NFS41_FSMount failed: Timeout Mike On Wed, Jan 18, 2017 at 9:35 AM, Russell L. Carter wrote: > On 01/18/17 09:44, Michael Ware wrote: > >> Just to give more info: >> rc.conf I have nfsv4_server_enable="yes" >> >> in /etc/exports >> V4: / -sec=sys -network 172.20.8.0/24 >> /export/vmimages -maproot=root -network 172.20.8.0/24 >> >> > This is what I use, no matter what subset of the filesystem > is being exported: > > / -maproot=root > V4: / -network 10.0.0.0/16 > > A surprisingly large amount of words exists in the lists > about why it can't be much more complicated than this. > > Russell > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > -- Michael Ware UCSC Baskin Engineering Unix, Network and Security 406-210-4725 From owner-freebsd-fs@freebsd.org Wed Jan 18 21:25:29 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C9FCCB62D5 for ; Wed, 18 Jan 2017 21:25:29 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: from h2.pinyon.org (h2.pinyon.org [65.101.20.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 498201DD9 for ; Wed, 18 Jan 2017 21:25:28 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: by h2.pinyon.org (Postfix, from userid 58) id 7450E5361; Wed, 18 Jan 2017 14:25:27 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=pinyon.org; s=DKIM; t=1484774727; bh=Y2w1K1qJQ9l2+YfrNZ83iNaYHXn1v+jXT7JIwpx0z08=; h=Subject:To:References:Cc:From:Date:In-Reply-To; b=dFJLvT4+kA7RelG7uv5iRCCvkimSOwVtC7EWvQjaiRr35ubmhV9Lm74uxz7IuAjLx xtgnMMVMnSl+dPgMpQNDeTvtme0Ezd0/aA+zzxjKlefky97WxCwx10W2PzToW+GL+Q fNKiDngJ470p29O5oFat7bAGOscKqLX/L/OWYdyo= X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on h2.pinyon.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,URIBL_BLOCKED shortcircuit=no autolearn=unavailable autolearn_force=no version=3.4.1 Received: from [10.0.10.15] (h1.pinyon.org [65.101.20.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by h2.pinyon.org (Postfix) with ESMTPSA id E1CC35349; Wed, 18 Jan 2017 14:25:25 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=pinyon.org; s=DKIM; t=1484774725; bh=Y2w1K1qJQ9l2+YfrNZ83iNaYHXn1v+jXT7JIwpx0z08=; h=Subject:To:References:Cc:From:Date:In-Reply-To; b=n9TjKnKKzgiZrdnE4ErwHDkA6g2Fh0+Yb3/M8CrqS4lhdDu3yg3NORAj4OVqt77HI r31sSWXhKW25mKB7wQJ7Su/8IVL/3+N9EBRBhUQQfdDFczCXa4ZK2yeih0n6BRuvbu m55wMlFmv+HIvFAnDNi311DuUK4lCyBxI2Xnw7XA= Subject: Re: NFS 4.1 exports To: Michael Ware References: <23392774-b72a-76c2-b637-6c592b504016@pinyon.org> Cc: "freebsd-fs@freebsd.org" From: "Russell L. Carter" Message-ID: Date: Wed, 18 Jan 2017 14:25:25 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 21:25:29 -0000 On 01/18/17 12:06, Michael Ware wrote: > So just for more info I tried to mount using the esxicli command with > the following results. > > [root@vmserver:~] esxcli storage nfs41 add -H storagearray.private -s > /export/vms -v vmstorage > Sysinfo error on operation returned status : Timeout. Please see the > VMkernel log for detailed error information > > 2017-01-18T19:00:27.779Z cpu4:2983706 opID=e9567d5d)NFS41: > NFS41_VSIMountSet:402: Mount server: storagearray.private, port: 2049, > path: /export/vms, label: vmstorage, security: 1 user: , options: > 2017-01-18T19:00:27.779Z cpu4:2983706 opID=e9567d5d)StorageApdHandler: > 982: APD Handle Created with lock[StorageApd-0x4308ab000130] > 2017-01-18T19:00:42.781Z cpu16:4324289)WARNING: SunRPC: 3947: fail all > pending calls for client 0x430440b90ee0 (socket disconnected) > 2017-01-18T19:00:47.779Z cpu0:2983706 opID=e9567d5d)WARNING: NFS41: > NFS41FSWaitForCluster:3433: Failed to wait for the cluster to be > located: Timeout > 2017-01-18T19:00:47.779Z cpu0:2983706 opID=e9567d5d)WARNING: NFS41: > NFS41_FSMount:4412: NFS41FSDoMount failed: Timeout > 2017-01-18T19:00:47.779Z cpu0:2983706 opID=e9567d5d)StorageApdHandler: > 1066: Freeing APD handle 0x4308ab000130 [] > 2017-01-18T19:00:47.779Z cpu0:2983706 opID=e9567d5d)StorageApdHandler: > 1150: APD Handle freed! > 2017-01-18T19:00:47.779Z cpu0:2983706 opID=e9567d5d)WARNING: NFS41: > NFS41_VSIMountSet:410: NFS41_FSMount failed: Timeout Rick Macklem is the goto person for this sort of thing. If you can test his patches that would be cool. Russell > Mike > > On Wed, Jan 18, 2017 at 9:35 AM, Russell L. Carter > wrote: > > On 01/18/17 09:44, Michael Ware wrote: > > Just to give more info: > rc.conf I have nfsv4_server_enable="yes" > > in /etc/exports > V4: / -sec=sys -network 172.20.8.0/24 > /export/vmimages -maproot=root -network 172.20.8.0/24 > > > > This is what I use, no matter what subset of the filesystem > is being exported: > > / -maproot=root > V4: / -network 10.0.0.0/16 > > A surprisingly large amount of words exists in the lists > about why it can't be much more complicated than this. > > Russell > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org > " > > > > > -- > Michael Ware > UCSC Baskin Engineering > Unix, Network and Security > 406-210-4725 From owner-freebsd-fs@freebsd.org Thu Jan 19 07:56:12 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4357CB76FD for ; Thu, 19 Jan 2017 07:56:12 +0000 (UTC) (envelope-from tobias.oberstein@gmail.com) Received: from mail-lf0-x22f.google.com (mail-lf0-x22f.google.com [IPv6:2a00:1450:4010:c07::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B6131443 for ; Thu, 19 Jan 2017 07:56:12 +0000 (UTC) (envelope-from tobias.oberstein@gmail.com) Received: by mail-lf0-x22f.google.com with SMTP id k86so29502245lfi.0 for ; Wed, 18 Jan 2017 23:56:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=uLj4aHt8zIOP8C/k0CuSg0OzlhXbaJftZx85jOVpkO0=; b=F6w69LZy2GMVJ8ilj/Ih+edFYxNlVi3dmdhOCIkIInJQC66a5sRv52nN/ETr+oP7+Y /pwxMGzmZBmwAeq+W+e9OC18dNivp2afHhxyCh/PuPusKlTT384b9yz+Uw7pRmUXkAva dNLi2/JqU30T4rCXhVJW7Tl/uanelLm66p+GuJ0K3GvjXPjecJMTpkdABC9avEijcJKM tV4eD7IEKZuiV+/ZZV3rdtzCFFBEAXFnxfwXWfKSDLWsGo9HRg7j5QB/yPHW8I2giWeU zfTp31kThja4vpQcjMd6p7/4NuHWt4LTaduFOucpnTHnKW5Slbt4ZOnxBjrHdHzkntu0 AyCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=uLj4aHt8zIOP8C/k0CuSg0OzlhXbaJftZx85jOVpkO0=; b=AWoUL2ia3sgKa2qk68tzD/VjG09HXY1C2k5miKRmrQkYA0hRDSS31l+1Z0IvUPkAGf tYGalILaVaJ3DFwN+6GdfOYaxZzXpcC8+pfThgXwbgjRgHLC9KkmTsch5hjaJ8B75dMz URDvtNz2tAxkfgPcgaVp4F++Uhs73J7oSFfYOIffbSZ3KjAsh27PmCKNgvXw9A9XCJTr EbyehS3lBIaccrKhNbrgRYTTTAX5u/Fzxx4WtFn6G351d7BUptEHFn3OH97a5OgLH41Q Q9W/m9FLcsE7PF3L/QqKu1fUGcYfB6j6LW4y1U0gDqxpc/7qXkFwGy34GDAGVksCqvO8 U9uA== X-Gm-Message-State: AIkVDXJEJezRTnVYllNoEyy4T6QhasBF1TE8526T/+sQyhDYNxQauCubffg0IGWa9xsXRA== X-Received: by 10.25.67.19 with SMTP id q19mr2536878lfa.9.1484812569207; Wed, 18 Jan 2017 23:56:09 -0800 (PST) Received: from [10.200.202.23] ([195.14.242.85]) by smtp.googlemail.com with ESMTPSA id 66sm1245561lfy.42.2017.01.18.23.56.08 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Jan 2017 23:56:08 -0800 (PST) To: freebsd-fs@freebsd.org From: Tobias Oberstein Subject: stuck on low scrub performance Message-ID: Date: Thu, 19 Jan 2017 08:56:05 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 07:56:12 -0000 Hi, I've got a 146TB RAID-Z2 magnetic platter pool which I am scrubbing, but performance is very poor: pool: tank state: ONLINE scan: scrub in progress since Tue Jan 17 11:13:29 2017 5.02T scanned out of 89.4T at 32.2M/s, (scan is slow, no estimated time) 0 repaired, 5.61% done I've already tried modifying various knobs, with no success (I changed oids while the scrub is running, without restarting - does that take effect?). I've tried to collect as much info as possible using this script https://gist.github.com/oberstet/c66baf777ea77ae843b64271df7420a3 with this output https://gist.github.com/oberstet/a8e12227486ebeb1e85f6f56b880ce7d --- My expectation from the general performance of this array would be scrubbing with 500MB/s (10x more) - this is what we regularily see on this array when it goes to the magnetic platters on sequential read loads. Any helps would be greatly appreciated, as at that speed, the scrubbing takes weeks =( Cheers, /Tobias From owner-freebsd-fs@freebsd.org Sat Jan 21 08:50:10 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4EE4CB77EE for ; Sat, 21 Jan 2017 08:50:10 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: from mail.sub.ru (mail.sub.ru [88.212.205.2]) by mx1.freebsd.org (Postfix) with SMTP id 3FE8A1284 for ; Sat, 21 Jan 2017 08:50:09 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: (qmail 75913 invoked from network); 21 Jan 2017 11:50:05 +0300 Received: from ppp109-252-90-166.pppoe.spdop.ru (ppp109-252-90-166.pppoe.spdop.ru [109.252.90.166]) by mail.sub.ru ([88.212.205.2]) with ESMTP via TCP; 31 Dec 1969 23:59:59 -0000 From: Alex Povolotsky Subject: Poor ZFS performance To: freebsd-fs@freebsd.org Message-ID: <595d8117-e2f2-fa4f-a45e-3a9fb93d0687@webmail.sub.ru> Date: Sat, 21 Jan 2017 11:49:50 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Antivirus: avast! (VPS 170120-2, 20.01.2017), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 08:50:11 -0000 Hello I'm writing lots of (tens of millions) relatively small files, hashing them out in three-level directory, 100 entries per level. Speed is quite unsatisfying, about 2 millions files per day writing in 20 processes, that is, about one file per second for process. CPU is not loaded (la 1.34, 0.4%sys, 0.2% intr, 5.7% user) Disk is used (sysat gives 0.45 MB/s, 89% busy) atime is off, even sync is disabled. FreeBSD insta1.inf.sudo.su 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 CPU: Intel(R) Xeon(R) CPU E5530 @ 2.40GHz (2400.14-MHz K8-class CPU) Origin="GenuineIntel" Id=0x106a5 Family=0x6 Model=0x1a Stepping=5 Features=0xbfebfbff Features2=0x9ce3bd AMD Features=0x28100800 AMD Features2=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,VPID TSC: P-state invariant, performance statistics real memory = 34363932672 (32772 MB) avail memory = 33271255040 (31729 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: <080312 APIC1521> FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 hardware threads No L2ARC. ARC Size: 60.73% 18.29 GiB Target Size: (Adaptive) 60.84% 18.32 GiB Min Size (Hard Limit): 12.50% 3.76 GiB Max Size (High Water): 8:1 30.12 GiB ARC Size Breakdown: Recently Used Cache Size: 93.56% 17.14 GiB Frequently Used Cache Size: 6.44% 1.18 GiB ARC Efficiency: 1.60b Cache Hit Ratio: 65.87% 1.05b Cache Miss Ratio: 34.13% 544.82m Actual Hit Ratio: 62.58% 999.04m Data Demand Efficiency: 99.88% 283.23m Data Prefetch Efficiency: 31.76% 109.26m (I've disabled prefetch yesterday with no visible result) Attempt to restart program with only two processes yields 10 files per second in several minutes falling to 5, to 3 and so on. Zpool status for the pool in question pool: Z840TZPB state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM Z840TZPB ONLINE 0 0 0 gpt/Z840TZPB ONLINE 0 0 0 errors: No known data errors smartctl === START OF INFORMATION SECTION === Model Family: Seagate Archive HDD Device Model: ST8000AS0002-1NA17Z Serial Number: Z840TZPB LU WWN Device Id: 5 000c50 093201557 Firmware Version: AR17 User Capacity: 8,001,563,222,016 bytes [8.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5980 rpm Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b SATA Version is: SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Sat Jan 21 11:41:30 2017 MSK SMART support is: Available - device has SMART capability. SMART support is: Enabled disk is fresh (420 hours), no errors. What can I do to improve writing performance? Alex From owner-freebsd-fs@freebsd.org Sat Jan 21 08:50:14 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A05DACB77FA for ; Sat, 21 Jan 2017 08:50:14 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8FA4F1288 for ; Sat, 21 Jan 2017 08:50:14 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: by mailman.ysv.freebsd.org (Postfix) id 8C0E6CB77F7; Sat, 21 Jan 2017 08:50:14 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BBC2CB77F6 for ; Sat, 21 Jan 2017 08:50:14 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: from mail.sub.ru (mail.sub.ru [88.212.205.2]) by mx1.freebsd.org (Postfix) with SMTP id CF3CE1286 for ; Sat, 21 Jan 2017 08:50:13 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: (qmail 72750 invoked from network); 21 Jan 2017 11:42:50 +0300 Received: from ppp109-252-90-166.pppoe.spdop.ru (ppp109-252-90-166.pppoe.spdop.ru [109.252.90.166]) by mail.sub.ru ([88.212.205.2]) with ESMTP via TCP; 31 Dec 1969 23:59:59 -0000 To: fs@freebsd.org From: Alex Povolotsky Subject: Poor ZFS performance Message-ID: <083391de-d153-e0f6-c453-63d95d3e1f55@webmail.sub.ru> Date: Sat, 21 Jan 2017 11:42:35 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Antivirus: avast! (VPS 170120-2, 20.01.2017), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 08:50:14 -0000 Hello I'm writing lots of (tens of millions) relatively small files, hashing them out in three-level directory, 100 entries per level. Speed is quite unsatisfying, about 2 millions files per day writing in 20 processes, that is, about one file per second for process. CPU is not loaded (la 1.34, 0.4%sys, 0.2% intr, 5.7% user) Disk is used (sysat gives 0.45 MB/s, 89% busy) atime is off, even sync is disabled. FreeBSD insta1.inf.sudo.su 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 CPU: Intel(R) Xeon(R) CPU E5530 @ 2.40GHz (2400.14-MHz K8-class CPU) Origin="GenuineIntel" Id=0x106a5 Family=0x6 Model=0x1a Stepping=5 Features=0xbfebfbff Features2=0x9ce3bd AMD Features=0x28100800 AMD Features2=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,VPID TSC: P-state invariant, performance statistics real memory = 34363932672 (32772 MB) avail memory = 33271255040 (31729 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: <080312 APIC1521> FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 hardware threads No L2ARC. ARC Size: 60.73% 18.29 GiB Target Size: (Adaptive) 60.84% 18.32 GiB Min Size (Hard Limit): 12.50% 3.76 GiB Max Size (High Water): 8:1 30.12 GiB ARC Size Breakdown: Recently Used Cache Size: 93.56% 17.14 GiB Frequently Used Cache Size: 6.44% 1.18 GiB ARC Efficiency: 1.60b Cache Hit Ratio: 65.87% 1.05b Cache Miss Ratio: 34.13% 544.82m Actual Hit Ratio: 62.58% 999.04m Data Demand Efficiency: 99.88% 283.23m Data Prefetch Efficiency: 31.76% 109.26m (I've disabled prefetch yesterday with no visible result) Attempt to restart program with only two processes yields 10 files per second in several minutes falling to 5, to 3 and so on. Zpool status for the pool in question pool: Z840TZPB state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM Z840TZPB ONLINE 0 0 0 gpt/Z840TZPB ONLINE 0 0 0 errors: No known data errors smartctl === START OF INFORMATION SECTION === Model Family: Seagate Archive HDD Device Model: ST8000AS0002-1NA17Z Serial Number: Z840TZPB LU WWN Device Id: 5 000c50 093201557 Firmware Version: AR17 User Capacity: 8,001,563,222,016 bytes [8.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5980 rpm Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b SATA Version is: SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Sat Jan 21 11:41:30 2017 MSK SMART support is: Available - device has SMART capability. SMART support is: Enabled disk is fresh (420 hours), no errors. What can I do to improve writing performance? Alex From owner-freebsd-fs@freebsd.org Sat Jan 21 09:28:06 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8C0ECB9785 for ; Sat, 21 Jan 2017 09:28:06 +0000 (UTC) (envelope-from cipher_nl@hotmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B7C351566 for ; Sat, 21 Jan 2017 09:28:06 +0000 (UTC) (envelope-from cipher_nl@hotmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id B718CCB9784; Sat, 21 Jan 2017 09:28:06 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6B84CB9783 for ; Sat, 21 Jan 2017 09:28:06 +0000 (UTC) (envelope-from cipher_nl@hotmail.com) Received: from BAY004-OMC3S13.hotmail.com (bay004-omc3s13.hotmail.com [65.54.190.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A7F41564 for ; Sat, 21 Jan 2017 09:28:06 +0000 (UTC) (envelope-from cipher_nl@hotmail.com) Received: from EUR03-AM5-obe.outbound.protection.outlook.com ([65.54.190.188]) by BAY004-OMC3S13.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Sat, 21 Jan 2017 01:27:00 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=tb/g5UcbO4rU9FETwv1/cEJNMIKp4S1mHRUC+8Khqhc=; b=khDFsl0eM8FnZjNboI5OYcXHfjShd5z/U4g1rCXOe2SDMJDJsWdHl4+i8sFt+t3Ec+5c0B9niZLAiP8RzEIvxhDNOzGeOmxzDrZCk14K6X+BXJcFeM6VTqqICoIGYQuqPU+m02Raj1y/Kdrs2gruDeyQkN/N2LJmKNvZAfrVzO+ltl9QEyv6egP1Ymbr9GPYCuihMdfb/dvXPyokFeq0uwMITElIdN7jpOB+a4/JeZnFUgXqfsqaKOUOIINtY1hJKGhYlS0Ne88HBqyBRYbonNxwLH89hxb+gec0rx/KeNXpgOt7DTVYWn1m18tZ7PzrI4Rkv+If7A6XG5OjiDgLsA== Received: from DB5EUR03FT052.eop-EUR03.prod.protection.outlook.com (10.152.20.55) by DB5EUR03HT180.eop-EUR03.prod.protection.outlook.com (10.152.21.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.803.8; Sat, 21 Jan 2017 09:26:58 +0000 Received: from DB6PR0801MB1622.eurprd08.prod.outlook.com (10.152.20.53) by DB5EUR03FT052.mail.protection.outlook.com (10.152.21.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.803.8 via Frontend Transport; Sat, 21 Jan 2017 09:26:57 +0000 Received: from DB6PR0801MB1622.eurprd08.prod.outlook.com ([10.169.226.10]) by DB6PR0801MB1622.eurprd08.prod.outlook.com ([10.169.226.10]) with mapi id 15.01.0860.018; Sat, 21 Jan 2017 09:26:58 +0000 From: "D. E" To: Alex Povolotsky , "fs@freebsd.org" Subject: Re: Poor ZFS performance Thread-Topic: Poor ZFS performance Thread-Index: AQHSc8N15Axkuc7NzUGbaTLlBQOz/KFCqWER Date: Sat, 21 Jan 2017 09:26:58 +0000 Message-ID: References: <083391de-d153-e0f6-c453-63d95d3e1f55@webmail.sub.ru> In-Reply-To: <083391de-d153-e0f6-c453-63d95d3e1f55@webmail.sub.ru> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: webmail.sub.ru; dkim=none (message not signed) header.d=none; webmail.sub.ru; dmarc=none action=none header.from=hotmail.com; x-incomingtopheadermarker: OriginalChecksum:E2C86382044B6F8F37172BDF5B3A4780B7A9A3198F20A249012C0E6E9E39F66C; UpperCasedChecksum:0443A63C591237F133EF18E03ECF580916FF17A49A5E7780AEB834884EAA9880; SizeAsReceived:7619; Count:39 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [z4/hnPJkhewMCYWoU1sxGI4RZaludDb7] x-incomingheadercount: 39 x-eopattributedmessage: 0 x-microsoft-exchange-diagnostics: 1; DB5EUR03HT180; 7:Mn6JE0TYj+zyGXC51qNiDuQ22lSJX47s4TvBueHMtNa1IP/8vj+qUk1f3gj3qI+TfbunPk8WZ+radXmgUQ4Hk7YKZ0Y2n6y3P2TYm5rj2IGcimv4c3L2iZKLZQKkforltQmwxpnBQ5MLMrX1169zx6ygN+HnLEL4/698lItcf3jZftPCSeDTSCtV/TR+o0etpSDbh6RWR2Ao0yD2Cf28snRsC9p2idcpJPOlGrNpAjZB+oYHZufOdaJtVjLdb0+h1hb4Wg80PPtqrnCc4TXCeKbLhimxco4d1Y9dzPqjJ9yPxmQ/pNoaBNAvU8Je3Ab7eheanfAxgEqkl2ioU0yMGfIJphAE/QXwH+TIQrRVqgUSlTgSGt5Cx35rmB9zNHo+SFjqY3BIwl3wG9t1gSUtRegRUCw3W3NktnaQhqzunA9HxdKWrhy8AXdi0APpQz8yI/Lmo9icQQFhGTVx9VFTIQ== x-forefront-antispam-report: EFV:NLI; SFV:NSPM; SFS:(10019020)(98900005); DIR:OUT; SFP:1102; SCL:1; SRVR:DB5EUR03HT180; H:DB6PR0801MB1622.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; x-ms-office365-filtering-correlation-id: 0139538b-ae7c-47b9-5d2e-08d441dfa602 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(1601124038)(5061506344)(1603103113)(1601125047)(1701031023); SRVR:DB5EUR03HT180; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(444111334)(444112120)(432015012)(82015046); SRVR:DB5EUR03HT180; BCL:0; PCL:0; RULEID:; SRVR:DB5EUR03HT180; x-forefront-prvs: 01949FE337 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jan 2017 09:26:58.2786 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5EUR03HT180 X-OriginalArrivalTime: 21 Jan 2017 09:27:00.0794 (UTC) FILETIME=[84ECF1A0:01D273C8] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 09:28:07 -0000 >I'm writing lots of (tens of millions) relatively small files, hashing=20 >them out in three-level directory, 100 entries per level. >Speed is quite unsatisfying, about 2 millions files per day writing in=20 >20 processes, that is, about one file per second for process. But that is to be expected, isn't it? You have chosen hardware which is *EX= TREMELY* slow for non-contiguous write access. The Seagate Archive HDD is a= n SMR harddrive suitable for archive storage - not a generic PMR harddrive.= So the Seagate Archive HDD is suitable for makings backups and writing hug= e files like 1TB images which you write from A to Z. SMR drives are notorio= usly slow when it comes to non-contiguous writes, such as writing small fil= es. You can read warnings about SMR and the Seagate Archive HDD in particular e= verywhere on the web. Have you missed them? Slightly exaggerated: your SMR-drive is more like a tapestreamer than a rea= l harddrive. It should be used to store enormous files and used for 'cold s= torage'. Read more about the Seagate Archive HDD on its website: http://www.seagate.= com/enterprise-storage/hard-disk-drives/archive-hdd/ To be honest, it is the worst harddrive one can buy today for the purpose o= f regular storage (hot storage). For cold storage, such as backups like tap= estreamers do the drive is decent, but not good by any means. 1,33TB per pl= atter is disappointing for SMR considering the massive downsides. And some = PMR drives are even cheaper per gigabyte than this SMR drive. So i strongly= recommend against SMR at this time. Regards, CiPHER= From owner-freebsd-fs@freebsd.org Sat Jan 21 09:47:09 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CACE5CB9DE3 for ; Sat, 21 Jan 2017 09:47:09 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B76EA1CC5 for ; Sat, 21 Jan 2017 09:47:09 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: by mailman.ysv.freebsd.org (Postfix) id B6BFBCB9DE2; Sat, 21 Jan 2017 09:47:09 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6655CB9DE1 for ; Sat, 21 Jan 2017 09:47:09 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: from mail.sub.ru (mail.sub.ru [88.212.205.2]) by mx1.freebsd.org (Postfix) with SMTP id 4AF571CC2 for ; Sat, 21 Jan 2017 09:47:08 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: (qmail 1123 invoked from network); 21 Jan 2017 12:47:04 +0300 Received: from ppp109-252-90-166.pppoe.spdop.ru (ppp109-252-90-166.pppoe.spdop.ru [109.252.90.166]) by mail.sub.ru ([88.212.205.2]) with ESMTP via TCP; 31 Dec 1969 23:59:59 -0000 Subject: Re: Poor ZFS performance To: "D. E" , "fs@freebsd.org" References: <083391de-d153-e0f6-c453-63d95d3e1f55@webmail.sub.ru> From: Alex Povolotsky Message-ID: Date: Sat, 21 Jan 2017 12:47:04 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Antivirus: avast! (VPS 170120-2, 20.01.2017), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 09:47:09 -0000 So fetching data to, say, ST2000NM0055-1V4104, and than zfs send | zfs receive to Archive HDD will do the trick? Seems like the answer is "yes", I've got 3-4 times speed improvement. In an hour, I'll try partial transfer. What 8 Tb disk would you recommend for raidz? On 21.01.2017 12:26, D. E wrote: >> I'm writing lots of (tens of millions) relatively small files, hashing >> them out in three-level directory, 100 entries per level. > >> Speed is quite unsatisfying, about 2 millions files per day writing in >> 20 processes, that is, about one file per second for process. > > But that is to be expected, isn't it? You have chosen hardware which is *EXTREMELY* slow for non-contiguous write access. The Seagate Archive HDD is an SMR harddrive suitable for archive storage - not a generic PMR harddrive. So the Seagate Archive HDD is suitable for makings backups and writing huge files like 1TB images which you write from A to Z. SMR drives are notoriously slow when it comes to non-contiguous writes, such as writing small files. > > You can read warnings about SMR and the Seagate Archive HDD in particular everywhere on the web. Have you missed them? > > Slightly exaggerated: your SMR-drive is more like a tapestreamer than a real harddrive. It should be used to store enormous files and used for 'cold storage'. > > Read more about the Seagate Archive HDD on its website: http://www.seagate.com/enterprise-storage/hard-disk-drives/archive-hdd/ > > To be honest, it is the worst harddrive one can buy today for the purpose of regular storage (hot storage). For cold storage, such as backups like tapestreamers do the drive is decent, but not good by any means. 1,33TB per platter is disappointing for SMR considering the massive downsides. And some PMR drives are even cheaper per gigabyte than this SMR drive. So i strongly recommend against SMR at this time. > > Regards, > CiPHER > From owner-freebsd-fs@freebsd.org Sat Jan 21 09:53:28 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46D74CBA107 for ; Sat, 21 Jan 2017 09:53:28 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 23B0710CC for ; Sat, 21 Jan 2017 09:53:28 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 2306ECBA106; Sat, 21 Jan 2017 09:53:28 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22A9CCBA105 for ; Sat, 21 Jan 2017 09:53:28 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AAE9D10CB for ; Sat, 21 Jan 2017 09:53:27 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22a.google.com with SMTP id r126so68797054wmr.0 for ; Sat, 21 Jan 2017 01:53:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=DC5mMgkqKlRNhsi4GAnIARK08X0XH/r5DXGRGInv63w=; b=qSK8AyYip31gk/zz/isOCkyrLAtL1v67X86xAVcERqI0xXUz5e3Dnj0Knk15h6VmRr BNsMFKK8Fedc9wTRwusFohzQSkPwh341963uFxRyOE+b0vzbdEwTbgBwUWBuIO6OdtDa WCTg+Vtn5XzdodxmE9OWM42y3qXG5vUkXrkj8XhFAUEWX52iU6ORbYbqVu/3bQ/cx2EA r0Y6MYapA0LFWQUFgv0kMWOZx5VtQvJek5jQW0/m5ElaHOGe6sGziy89K4dwEhGT3/+/ M7skTAftJggwHPyvKQI3ifu9FkhyP5W4WyZToSHz0Mouv02xCqkDq4r4L9CFdFXS01Ap JIVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=DC5mMgkqKlRNhsi4GAnIARK08X0XH/r5DXGRGInv63w=; b=cdBI9oC05ar7/X6xytXMAElYeQg4mesv7pTo8uewcDH8hrHsDAG8KvHXZ59WXIDq6z lwdC1GaeXVRgV6ez++AX5Qf54PVF7a6D1AZXA/TVm2OQX0HsRkzqmR0tY4XciReDsXbU jWUoHFV0lgqWegqphqL5UyriKZeIwdRmy3fMbRFz21FfYyrdPBhRy/pvjAIlVOu0nolK InN9G163Zbhl1vr/VuXusiL/ZPJskedB1gPPtxbyoClnbjKCYztXyvgEvkHx6bNUNVwy jfa+FkKgCTCK5+WsoAly4QVaGyltWx15dorW0GEPosbLqQf5m0oCBs5aYczwQpJMofJz 9JgQ== X-Gm-Message-State: AIkVDXLC1cOwIg0ZlVDtHT8pFbAVhQjP0kv2mp1ckYIOfyBr4cyTIjmprSP/2vZaw2Njlg== X-Received: by 10.28.129.147 with SMTP id c141mr7151307wmd.12.1484992405189; Sat, 21 Jan 2017 01:53:25 -0800 (PST) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id s26sm3668987wra.26.2017.01.21.01.53.24 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 21 Jan 2017 01:53:24 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Poor ZFS performance From: Ben RUBSON In-Reply-To: Date: Sat, 21 Jan 2017 10:53:23 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <963575CE-F74A-456A-AB33-3C5F6828ED17@gmail.com> References: <083391de-d153-e0f6-c453-63d95d3e1f55@webmail.sub.ru> To: "fs@freebsd.org" X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 09:53:28 -0000 Hi, What about a primary pool made of SSD ? They should fit your workload much better than platter disks. Ben > On 21 Jan 2017, at 10:47, Alex Povolotsky = wrote: >=20 > So fetching data to, say, ST2000NM0055-1V4104, and than zfs send | zfs = receive to Archive HDD will do the trick? >=20 > Seems like the answer is "yes", I've got 3-4 times speed improvement. = In an hour, I'll try partial transfer. >=20 > What 8 Tb disk would you recommend for raidz? >=20 > On 21.01.2017 12:26, D. E wrote: >>> I'm writing lots of (tens of millions) relatively small files, = hashing >>> them out in three-level directory, 100 entries per level. >>=20 >>> Speed is quite unsatisfying, about 2 millions files per day writing = in >>> 20 processes, that is, about one file per second for process. >>=20 >> But that is to be expected, isn't it? You have chosen hardware which = is *EXTREMELY* slow for non-contiguous write access. The Seagate Archive = HDD is an SMR harddrive suitable for archive storage - not a generic PMR = harddrive. So the Seagate Archive HDD is suitable for makings backups = and writing huge files like 1TB images which you write from A to Z. SMR = drives are notoriously slow when it comes to non-contiguous writes, such = as writing small files. >>=20 >> You can read warnings about SMR and the Seagate Archive HDD in = particular everywhere on the web. Have you missed them? >>=20 >> Slightly exaggerated: your SMR-drive is more like a tapestreamer than = a real harddrive. It should be used to store enormous files and used for = 'cold storage'. >>=20 >> Read more about the Seagate Archive HDD on its website: = http://www.seagate.com/enterprise-storage/hard-disk-drives/archive-hdd/ >>=20 >> To be honest, it is the worst harddrive one can buy today for the = purpose of regular storage (hot storage). For cold storage, such as = backups like tapestreamers do the drive is decent, but not good by any = means. 1,33TB per platter is disappointing for SMR considering the = massive downsides. And some PMR drives are even cheaper per gigabyte = than this SMR drive. So i strongly recommend against SMR at this time. >>=20 >> Regards, >> CiPHER >>=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Sat Jan 21 10:03:59 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D701BCBA580 for ; Sat, 21 Jan 2017 10:03:59 +0000 (UTC) (envelope-from jordanhubbard@icloud.com) Received: from pv35p22im-ztdg05131101.me.com (pv35p22im-ztdg05131101.me.com [17.133.189.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B9D63155F for ; Sat, 21 Jan 2017 10:03:59 +0000 (UTC) (envelope-from jordanhubbard@icloud.com) Received: from process-dkim-sign-daemon.pv35p22im-ztdg05131101.me.com by pv35p22im-ztdg05131101.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) id <0OK400200H06QU00@pv35p22im-ztdg05131101.me.com> for freebsd-fs@freebsd.org; Sat, 21 Jan 2017 09:03:53 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=4d515a; t=1484989433; bh=Zy5BKbTv2jfluEndkDRpty3LC147ZELzFhklCnanrDI=; h=From:Message-id:Content-type:MIME-version:Subject:Date:To; b=hkPUyrkjS3vz6tbbQXvRSvvpErN+ZRBtcPr+yddbAzu+ngvkWHAaabiN/dWMUhrc/ WJ1o60Dk+oCetr5EngkNgvirEYo7L2dVTPTj+McL22kAZwYWXqcyeKMASVvWORxa+d HoRWFKQEEKTmiTXETYa57IP8w+buZ1o+7oJdrg77fvKCImciwyiDx+cI1L/ML5VTAZ XNpW9l3z0o/LSma8MI0YLYX4RCzC6OteCsFeVaWji7A7ZIazypA4rawtKVvq6bziKd QUSG41DsbMw6Ka7rTeOthzBNDCr+dRCaeRSrfWfrvUot7iHfffPrHznsiGv1GtUVmx cOu+KkIen6sgg== Received: from [10.11.111.236] (50-250-239-90-static.hfc.comcastbusiness.net [50.250.239.90]) by pv35p22im-ztdg05131101.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) with ESMTPSA id <0OK4002A5H6FYR00@pv35p22im-ztdg05131101.me.com>; Sat, 21 Jan 2017 09:03:52 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-01-21_06:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1034 suspectscore=18 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1603290000 definitions=main-1701210134 From: Jordan Hubbard Message-id: MIME-version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: Poor ZFS performance Date: Sat, 21 Jan 2017 01:03:51 -0800 In-reply-to: <595d8117-e2f2-fa4f-a45e-3a9fb93d0687@webmail.sub.ru> Cc: freebsd-fs@freebsd.org To: Alex Povolotsky References: <595d8117-e2f2-fa4f-a45e-3a9fb93d0687@webmail.sub.ru> X-Mailer: Apple Mail (2.3259) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 10:03:59 -0000 > On Jan 21, 2017, at 12:49 AM, Alex Povolotsky > wrote: >=20 > I'm writing lots of (tens of millions) relatively small files, hashing = them out in three-level directory, 100 entries per level. You=E2=80=99re never going to get good performance doing that. The = constraints placed on ZFS=E2=80=99 design for directories (UNIX API, = POSIX compliance, etc) and small file representation formats means = it=E2=80=99ll never be a =E2=80=9Cdatabase=E2=80=9D - the filesystem's = design, to say nothing of UNIX=E2=80=99s directory iteration / lookup = APIs, just aren't optimized for millions of small files because it was = never the intention that any UNIX filesystem be a low-cost KVS or = database analog. Things will get quickly pathological from a = performance perspective and fixing the pathologies would require such a = significant redesign of a number of different pieces of the puzzle here = that it=E2=80=99s never likely to happen. Your application would be far better served by using an actual database. = I=E2=80=99m not just suggesting this as a hypothetical, either. I=E2=80=99= ve dealt with several folks who went down this path, storing millions = aornd even billions of small files into ZFS, and the results have never = been pretty, nor have there been any easy options or =E2=80=9Csimple = tunables=E2=80=9D that were going to make those scenarios significantly = prettier. The advice was the same: This needs to be a database, and = there are plenty of those to choose from with all kinds of performance / = consistency / redundancy characteristics to pick and choose between. - Jordan