From owner-freebsd-fs@freebsd.org Sun Aug 20 05:05:21 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF7A5DDFD9F for ; Sun, 20 Aug 2017 05:05:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BDBD67391F for ; Sun, 20 Aug 2017 05:05:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v7K55LSN007675 for ; Sun, 20 Aug 2017 05:05:21 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 215634] zfs receive trips up and live-locks for non-incremental fs Date: Sun, 20 Aug 2017 05:05:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: allanjude@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Aug 2017 05:05:21 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215634 Allan Jude changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |allanjude@FreeBSD.org, | |avg@FreeBSD.org, | |jpaetzel@FreeBSD.org --- Comment #1 from Allan Jude --- I think I hit this same thing doing a non-incremental send from FreeBSD 11.= 1 to FreeBSD 10.3: PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root 8 155 ki31 0K 128K CPU7 7 ??? 600.00% [i= dle] 3 root 8 -8 - 0K 144K CPU4 4 362:30 199.76% [zfskern] The 'zfs recv' command is stuck on the waitchan: rwa.cv load: 2.97 cmd: zfs 779 [rwa.cv] 85.90r 0.00u 0.41s 0% 2876k load: 2.40 cmd: zfs 779 [rwa.cv] 1049.93r 0.00u 0.41s 0% 2876k #procstat -kk 78197 779 PID TID COMM TDNAME KSTACK 78197 103920 zfs - mi_switch+0xe1 sleepq_wait+0= x3a _cv_wait+0x17d dmu_recv_stream+0x9db zfs_ioc_recv+0x96d zfsdev_ioctl+0x63d devfs_ioctl_f+0x139 kern_ioctl+0x255 sys_ioctl+0x140 amd64_syscall+0x40f Xfast_syscall+0xfb 779 102626 zfs - mi_switch+0xe1 sleepq_wait+0= x3a _cv_wait+0x17d dmu_recv_stream+0x9db zfs_ioc_recv+0x96d zfsdev_ioctl+0x63d devfs_ioctl_f+0x139 kern_ioctl+0x255 sys_ioctl+0x140 amd64_syscall+0x40f Xfast_syscall+0xfb --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sun Aug 20 05:06:11 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4520DDFE7C for ; Sun, 20 Aug 2017 05:06:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A2C86739DF for ; Sun, 20 Aug 2017 05:06:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v7K56BVe009181 for ; Sun, 20 Aug 2017 05:06:11 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 215634] zfs receive trips up and live-locks for non-incremental fs Date: Sun, 20 Aug 2017 05:06:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: allanjude@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Aug 2017 05:06:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215634 --- Comment #2 from Allan Jude --- procstat for zfskern: #procstat -kk 3 PID TID COMM TDNAME KSTACK 3 100069 zfskern arc_reclaim_thre mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x19e arc_reclaim_thread+0x28d fork_exit+0x9a fork_trampoline+0xe 3 100070 zfskern arc_user_evicts_ mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x19e arc_user_evicts_thread+0x13c fork_exit+0x9a fork_trampoline+0xe 3 100071 zfskern l2arc_feed_threa mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x19e l2arc_feed_thread+0x16f fork_exit+0x9a fork_trampoline+0xe 3 100375 zfskern trim zstore mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x19e trim_thread+0x9e fork_exit+0x= 9a fork_trampoline+0xe 3 100417 zfskern txg_thread_enter mi_switch+0xe1 sleepq_wait+0= x3a _cv_wait+0x17d txg_quiesce_thread+0x1ab fork_exit+0x9a fork_trampoline+0xe 3 100418 zfskern txg_thread_enter mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x19e txg_sync_thread+0x1dc fork_exit+0x9a fork_trampoline+0xe 3 101617 zfskern solthread 0xffff 3 101772 zfskern solthread 0xffff --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sun Aug 20 12:35:25 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76A99DCFB1B for ; Sun, 20 Aug 2017 12:35:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 63EF53DD6 for ; Sun, 20 Aug 2017 12:35:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v7KCZPnN018051 for ; Sun, 20 Aug 2017 12:35:25 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 220546] [patch] zfs.ko: undefined reference to abd_is_linear(), abd_copy_to_buf() and (probably) others with ASSERT's enabled Date: Sun, 20 Aug 2017 12:35:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avos@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Aug 2017 12:35:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220546 Andriy Voskoboinyk changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|New |Closed --- Comment #2 from Andriy Voskoboinyk --- Committed in r322601. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Aug 21 17:21:31 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54D8FDE9284 for ; Mon, 21 Aug 2017 17:21:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 42E7D7C646 for ; Mon, 21 Aug 2017 17:21:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v7LHLVwm049880 for ; Mon, 21 Aug 2017 17:21:31 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 221188] zfs: bin/chmod/chmod_test:v_flag fails on ZFS, not UFS (ZFS updates mode for files unnecessarily) Date: Mon, 21 Aug 2017 17:21:31 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Aug 2017 17:21:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221188 --- Comment #7 from commit-hook@freebsd.org --- A commit references this bug: Author: gjb Date: Mon Aug 21 17:20:31 UTC 2017 New revision: 322759 URL: https://svnweb.freebsd.org/changeset/base/322759 Log: MFC r321949, r321950, r322101: r321949 (ngie): Add expected failures for ZFS - :f_flag fails on ZFS because UF_IMMUTABLE isn't supported. - :v_flag fails on ZFS because the mode for foo is [always] updated unnecessarily. get_filesystem(..) (supporting function that was added to the test script) is based on equivalent logic in usr.bin/extattr/tests/extattr_test.sh . r321950 (ngie): Always use first parameter passed to get_filesystem(..) instead of discarding it and using `.` instead. r322101 (ngie): Don't check result of chflags in f_flag_cleanup() This will prevent false positives from occurring if the test is run on ZFS since ZFS doesn't support fflags throbbing like UFS. PR: 221188, 221189 Approved by: re (marius) Sponsored by: The FreeBSD Foundation Changes: _U stable/10/ stable/10/bin/chmod/tests/chmod_test.sh --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Aug 21 17:21:33 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D4ACDE928D for ; Mon, 21 Aug 2017 17:21:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B76E7C64D for ; Mon, 21 Aug 2017 17:21:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v7LHLWEp050274 for ; Mon, 21 Aug 2017 17:21:32 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 221189] zfs: bin/chmod/chmod_test:f_flag fails on ZFS, not UFS; UF_IMMUTABLE not supported on ZFS Date: Mon, 21 Aug 2017 17:21:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Aug 2017 17:21:33 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221189 --- Comment #9 from commit-hook@freebsd.org --- A commit references this bug: Author: gjb Date: Mon Aug 21 17:20:31 UTC 2017 New revision: 322759 URL: https://svnweb.freebsd.org/changeset/base/322759 Log: MFC r321949, r321950, r322101: r321949 (ngie): Add expected failures for ZFS - :f_flag fails on ZFS because UF_IMMUTABLE isn't supported. - :v_flag fails on ZFS because the mode for foo is [always] updated unnecessarily. get_filesystem(..) (supporting function that was added to the test script) is based on equivalent logic in usr.bin/extattr/tests/extattr_test.sh . r321950 (ngie): Always use first parameter passed to get_filesystem(..) instead of discarding it and using `.` instead. r322101 (ngie): Don't check result of chflags in f_flag_cleanup() This will prevent false positives from occurring if the test is run on ZFS since ZFS doesn't support fflags throbbing like UFS. PR: 221188, 221189 Approved by: re (marius) Sponsored by: The FreeBSD Foundation Changes: _U stable/10/ stable/10/bin/chmod/tests/chmod_test.sh --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Aug 21 17:26:23 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 42C07DE9641 for ; Mon, 21 Aug 2017 17:26:23 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [46.17.166.21]) by mx1.freebsd.org (Postfix) with ESMTP id C70087C8E0 for ; Mon, 21 Aug 2017 17:26:21 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (higson.cam.lispworks.com [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.15.2/8.15.2) with ESMTP id v7LHErAb046174; Mon, 21 Aug 2017 18:14:53 +0100 (BST) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (localhost.localdomain [127.0.0.1]) by higson.cam.lispworks.com (8.14.4) id v7LHEr0F010633; Mon, 21 Aug 2017 18:14:53 +0100 Received: (from martin@localhost) by higson.cam.lispworks.com (8.14.4/8.14.4/Submit) id v7LHErBa010629; Mon, 21 Aug 2017 18:14:53 +0100 Date: Mon, 21 Aug 2017 18:14:53 +0100 Message-Id: <201708211714.v7LHErBa010629@higson.cam.lispworks.com> From: Martin Simmons To: Chris Ross CC: freebsd-fs@freebsd.org In-reply-to: <4BCBA5DE-6AC7-4B0A-BF33-CFFE5A54B2DF@distal.com> (message from Chris Ross on Fri, 18 Aug 2017 23:30:49 -0400) Subject: Re: zfs recv went idle after 20+ hours References: <4BCBA5DE-6AC7-4B0A-BF33-CFFE5A54B2DF@distal.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Aug 2017 17:26:23 -0000 >>>>> On Fri, 18 Aug 2017 23:30:49 -0400, Chris Ross said: > > From earlier posts on this list, many know I was dealing with an issue on a zfs pool. I eventually got it running in an off way, and backed it up (via zfs send to a file in UFS). I then destroyed and recreated the pool as I wanted it. For the last 23 hours or so, I’ve been running a “zfs recv” from the large send output (~5.2T). It was cooking along, more slowly than I’d expected, but moving. > > I looked in on it a short while ago, and the pv I’m using on the input file was reporting 0.0 B/s, 77% complete. Looking at the iostat I had running, I could see the raidz disks still seeing activity, but less than a minute later that stopped. Now, all disks on the system seem idle. The “zfs recv” process is idle according to ps (STAT I+), and a ktrace on the zfs process shows no activity either. The window in which I was running the commands look as follows: > > cross@hyrule[~](511): zpool list > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > tank 18.8T 949K 18.8T - 0% 0% 1.00x ONLINE - > zroot 50.5G 34.7G 15.8G - 55% 68% 1.00x ONLINE - > cross@hyrule[~](512): pv /mnt/tank@20170815| sudo zfs recv -Fvu tank > receiving full stream of tank@20170815 into tank@20170815 > received 46.3KB stream in 1 seconds (46.3KB/sec) > receiving full stream of tank/deluge@20170815 into tank/deluge@20170815 > received 2.87TB stream in 45452 seconds (66.3MB/sec) ] 55% ETA 10:18:14 > receiving full stream of tank/timecapsule@20170815 into tank/timecapsule@20170815 > 3.99TiB 22:54:18 [0.00 B/s] [========================> ] 76% ETA 7:04:49 > > The time-of-day and ETA value keep increasing, but other than that, it’s just sitting. zfs list shows: > > hyrule# zfs list -t all | grep tank > tank 3.89T 8.28T 73.3K none > tank@20170815 0 - 73.3K - > tank/deluge 2.86T 8.28T 2.86T /data/deluge > tank/deluge@20170815 0 - 2.86T - > tank/timecapsule 1024G 8.28T 1024G /data/timecapsule > > I’m going to leave it this way for a while, but it’s already been about 10 minutes since it started, so I don’t expect that it will free itself. Does anyone have a suggestion, or something I should look at? Did I do something wrong? Check that top and/or ps show nothing else using CPU. You could use procstat -kk to see what each unexpectedly idle process is waiting for in the kernel. __Martin From owner-freebsd-fs@freebsd.org Mon Aug 21 18:28:08 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46D24DECF9F for ; Mon, 21 Aug 2017 18:28:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 352FC7F325 for ; Mon, 21 Aug 2017 18:28:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v7LIS640052332 for ; Mon, 21 Aug 2017 18:28:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 215634] zfs receive trips up and live-locks for non-incremental fs Date: Mon, 21 Aug 2017 18:28:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Aug 2017 18:28:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215634 --- Comment #3 from Andriy Gapon --- (In reply to Allan Jude from comment #2) It would be interesting to see where the spinning threads actually spin. Either DTrace profiling or hwpmc could be used for that. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Aug 22 09:26:42 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ECD0DDDC05C for ; Tue, 22 Aug 2017 09:26:42 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B1FF57DBAB for ; Tue, 22 Aug 2017 09:26:42 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1dk5Cf-0000Ag-NO; Tue, 22 Aug 2017 11:10:30 +0200 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: "freebsd-fs@freebsd.org" , "Rick Macklem" Subject: Re: when has a pNFS data server failed? References: Date: Tue, 22 Aug 2017 11:10:29 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.2 X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 autolearn=disabled version=3.4.0 X-Scan-Signature: 1f72ff50073f138f9668c095d6f579a1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2017 09:26:43 -0000 On Fri, 18 Aug 2017 23:52:12 +0200, Rick Macklem wrote: > This is kind of a "big picture" question that I thought I 'd throw out. > > As a brief background, I now have the code for running mirrored pNFS > Data Servers > working for normal operation. You can look at: > http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt > if you are interested in details related to the pNFS server code/testing. > > So, now I am facing the interesting part: > 1 - The Metadata Server (MDS) needs to decide that a mirrored DS has > failed at some > point. Once that happens, it stops using the DS, etc. > --> This brings me to the question of "when should the MDS decide that > the DS has > failed and should be taken offline?". > - I'm not up to date w.r.t. the TCP stack, so I'm not sure how > long it will take for the > TCP connection to decide that a DS server is no longer working > and fail the TCP > connection. I think it takes a fair amount of time, so I'm not > sure if TCP connection > loss is a good indicator of DS server failure or not? > - It seems to me that the MDS should wait a fairly long time before > failing the DS, > since this will have a major impact on the pNFS server, requiring > repair/resilvering > by a sysadmin once it happens. > So, any comments or thoughts on this? rick Hi, This is a quite common problem for all clustered/connected systems. I think there is no general answer. And there are a lot of papers written about it. For example: in NFS you have the 'soft' option. It is recommended not to use it. I can imagine that if your home-dir or /usr is mounted over NFS, but at work I want my http-servers to not hang and just give an IO-error when the backend fileserver with data is gone. Something similar happens here. Doesn't the protocol definition say something about this? Or what do other implemenations do? Regards, Ronald. From owner-freebsd-fs@freebsd.org Tue Aug 22 11:38:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 534B5DE40AA for ; Tue, 22 Aug 2017 11:38:05 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 293AF81F6A for ; Tue, 22 Aug 2017 11:38:04 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 2992A20BCE for ; Tue, 22 Aug 2017 07:38:03 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Tue, 22 Aug 2017 07:38:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=1GmQhsFDtSMt537kTU CIiBkYErvEhiXkYmq9+IURroQ=; b=nReHfBY8n3cQt5eINx2CHDLYvNyzFuZ88D hVwC1e6qaLafsU1RMIWlkcdGhWD3UEeMMOFpokRJhETeBIPoiW58Pg9KjQPutdF3 0Atm9gVChwsu4lxebKNomJ2bapR+sw31dmDW9U9F2BU/2Fg7EreyxOH9XNbejoP5 tWcIxFvaL8I8cHTEFaQwSpFS0L3pHjwqpXs67ixz8SQQxEKBE2HBRDbcil9K9QG+ pDkgBnA3rBbOKPqYRI+qlVut3xt4SLWR9NxaAsFIEgcpqJwTpxxJoSDpPydHVgAo 5+/INY0YOTIXzg1KyuMiAJrUYfZDTVAYc6Mb+qhQTBVQq0z/ctxQ== X-ME-Sender: X-Sasl-enc: 9G7uaXBuS8xpHJ3fAKDhLWVN3yLcomupKvUdy6e7bOYl 1503401882 Received: from ebox.rath.org (ebox.rath.org [45.79.69.51]) by mail.messagingengine.com (Postfix) with ESMTPA id E0EDD7E6AD for ; Tue, 22 Aug 2017 07:38:02 -0400 (EDT) Received: from vostro.rath.org (vostro [192.168.12.4]) by ebox.rath.org (Postfix) with ESMTPS id E68D4D5 for ; Tue, 22 Aug 2017 11:38:01 +0000 (UTC) Received: by vostro.rath.org (Postfix, from userid 1000) id 7223F1031FD; Tue, 22 Aug 2017 13:38:00 +0200 (CEST) From: Nikolaus Rath To: freebsd-fs@freebsd.org Subject: Can telldir() == 0 value be made special? Mail-Copies-To: never Mail-Followup-To: freebsd-fs@freebsd.org Date: Tue, 22 Aug 2017 13:38:00 +0200 Message-ID: <87bmn7kf3b.fsf@vostro.rath.org> User-Agent: Gnus/5.130014 (Ma Gnus v0.14) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2017 11:38:05 -0000 Hello, I am trying to debug a test failure of libfuse under FreeBSD. I believe I have reduced it to the following root cause: Consider the following program: #include #include #include int main(void) { struct dirent *e =3D NULL; DIR *dirp; printf("opendir...\n"); dirp =3D opendir("/"); if(dirp =3D=3D NULL) { perror("opendir"); return 1; } printf("telldir: %ld\n", telldir(dirp)); e =3D readdir(dirp); printf("readdir: %s\n", e->d_name); printf("telldir: %ld\n", telldir(dirp)); e =3D readdir(dirp); printf("readdir: %s\n", e->d_name); printf("closedir..\n"); closedir(dirp); =20=20=20=20 printf("opendir...\n"); dirp =3D opendir("/"); if(dirp =3D=3D NULL) { perror("opendir"); return 1; } e =3D readdir(dirp); printf("readdir: %s\n", e->d_name); printf("telldir: %ld\n", telldir(dirp)); e =3D readdir(dirp); printf("readdir: %s\n", e->d_name); printf("closedir..\n"); closedir(dirp); =20=20=20=20 return 0; } Under FreeBSD, running it gives: # ./simple=20 opendir... telldir: 0 readdir: . telldir: 1 readdir: .. closedir.. opendir... readdir: . telldir: 0 readdir: .. closedir.. In other words, if telldir() is called right after opendir(), it gives an offset of zero. But if telldir() is called only after the first readdir() call, it also gives an offset of zero. My hypothesis is that FreeBSD actually just enumerates the different telldir() calls - is that correct? Now, the offsets returned by telldir() are documented to be valid only within a given *dirp, so FreeBSD isn't doing anything wrong. However, having different meanings even for an offset of zero causes problems for libfuse, because under Linux an offset of zero is guaranteed to mean "first entry". This is reflected in the definition of the fuse readdir() function which always receives an *offset* parameter that needs to have a definite value even when telldir() was never called. If zero is suddenly also a valid telldir() return value that may indicate some other position in the stream, things get complicated. Now, I think I managed to work around that by shifting all offsets by one, but that is awkward (and I may have overlooked some problems that the unit tests don't cover). So I am wondering: Is there a reason why the FreeBSD kernel could not start enumerating telldir() offsets with 1, so that 0 can always have the same meaning? Best, -Nikolaus --=20 GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2= =AB From owner-freebsd-fs@freebsd.org Tue Aug 22 14:14:29 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20DFBDEDE15 for ; Tue, 22 Aug 2017 14:14:29 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (unknown [IPv6:2001:470:1f0f:3ad:bb:dcff:fe50:d900]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 026723E2D for ; Tue, 22 Aug 2017 14:14:28 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Content-Type:MIME-Version:Message-ID:Subject:To:From:Date:Sender :Reply-To:Cc:Content-Transfer-Encoding:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=TRtHfSmSSim3toG7dn1RaUcTYOnuRRV4ibtpvir6O3s=; b=kAQ+JBqncvcrVQy9Y2GujPCV7b 1D5jG/c6jrROvN3Q73u2JnNGWojVmCE5GxUWqwCNykQ1HRRziHBkPKHVDrbHKeeBPNX7ST/+4f34F wHrl9juv/B1Nw7sSxqXb6zuAD1uxJPmnOJqonMh6VLHdKSituORBfM9LeqiAPtZZMo8A=; Received: from [74.203.163.58] (port=33822 helo=lrosenman.local) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89 (FreeBSD)) (envelope-from ) id 1dk9wq-0003L2-0Q for freebsd-fs@FreeBSD.org; Tue, 22 Aug 2017 09:14:28 -0500 Date: Tue, 22 Aug 2017 09:14:19 -0500 From: Larry Rosenman To: freebsd-fs@FreeBSD.org Subject: ENAMETOOLONG? Message-ID: <20170822141419.v6ely424bjvcnzoj@lrosenman.local> Mail-Followup-To: freebsd-fs@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: NeoMutt/20170714 (1.8.3) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2017 14:14:29 -0000 Are there plans to increase the max length for zfs? [1239928] ZFS WARNING: Unable to create ZVOL zroot/vm-bhyve/freebsd11/disk0@zfs-auto-snap_hourly-2017-08-22-09h00 (error=63). FreeBSD thebighonker.lerctr.org 11.1-STABLE FreeBSD 11.1-STABLE #0 r321618: Thu Jul 27 08:21:09 CDT 2017 root@thebighonker.lerctr.org:/usr/obj/usr/src/sys/GENERIC amd64 1101501 1101501 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106 From owner-freebsd-fs@freebsd.org Tue Aug 22 14:42:45 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2156DEF6C1 for ; Tue, 22 Aug 2017 14:42:45 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-oi0-f46.google.com (mail-oi0-f46.google.com [209.85.218.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 60A3064194 for ; Tue, 22 Aug 2017 14:42:44 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-oi0-f46.google.com with SMTP id j144so59896650oib.1 for ; Tue, 22 Aug 2017 07:42:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:content-transfer-encoding; bh=ba+jaIp0qAjRFNlK1YPVMk7/oooKothTuksgPO4sGyU=; b=PcyDD+YidGnTHLLLzcrG0RufNgY9dvCSnWeaBRK9kwv32hPiTycHtb1lnrZDlaufVM 7/2d5WwDH3zVTn3Ifqfbn7FREGKFHv2EX/dF0VdcsH5uij+xBUIB6kRPM9qTZMbAabsd 1cI0l/yyKxVe4ZSd8Np5ihMDZD7jtbnnF5T+bXJe4s6W4Q+RBwlG5Gw7LH5hoOFh/pKn Dv8CjiRAymUqVyfBwr9nxgSbn1fuZDoFEothUMPQ5yhYmzkUId9eQ2BOjkvd0yIFOvGO xoo3xyB9I+oJDpCZ+kfhY4TycBdJmNnRoItuEXnlIEFdZUgvgK2IeHjh2zfOHLDt6NES j2zA== X-Gm-Message-State: AHYfb5iRkGuHpPIG8qyz+IeAB94TIAYAThe8oXj/MfIQfPNBHu2zu/jm 0gRj0fAsIm8MNPR52lC87w== X-Received: by 10.202.190.196 with SMTP id o187mr1121926oif.264.1503412957446; Tue, 22 Aug 2017 07:42:37 -0700 (PDT) Received: from mail-io0-f176.google.com (mail-io0-f176.google.com. [209.85.223.176]) by smtp.gmail.com with ESMTPSA id d95sm3004350oic.14.2017.08.22.07.42.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 Aug 2017 07:42:37 -0700 (PDT) Received: by mail-io0-f176.google.com with SMTP id y4so16339099iod.4 for ; Tue, 22 Aug 2017 07:42:37 -0700 (PDT) X-Received: by 10.107.185.196 with SMTP id j187mr774775iof.248.1503412956784; Tue, 22 Aug 2017 07:42:36 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.2.81.131 with HTTP; Tue, 22 Aug 2017 07:42:36 -0700 (PDT) In-Reply-To: <87bmn7kf3b.fsf@vostro.rath.org> References: <87bmn7kf3b.fsf@vostro.rath.org> From: Conrad Meyer Date: Tue, 22 Aug 2017 07:42:36 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Can telldir() == 0 value be made special? To: freebsd-fs@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2017 14:42:45 -0000 Hi Nikolaus, As you have surmised, DIR* seekpoints are created dynamically whenever requested by user's telldir() call: https://github.com/freebsd/freebsd/blob/master/lib/libc/gen/telldir.c#L53 I believe we could special case zero without breaking ABI compatibility of correct programs. Something like this: --- a/lib/libc/gen/telldir.c +++ b/lib/libc/gen/telldir.c @@ -70,6 +70,20 @@ telldir(DIR *dirp) } } if (lp =3D=3D NULL) { + /* Create special zero telldir entry, similar to Linux */ + if (dirp->dd_td->td_loccnt =3D=3D 0 && dirp->dd_loc !=3D 0)= { + lp =3D malloc(sizeof(struct ddloc)); + if (lp =3D=3D NULL) { + if (__isthreaded) + _pthread_mutex_unlock(&dirp->dd_loc= k); + return (-1); + } + lp->loc_index =3D dirp->dd_td->td_loccnt++; + lp->loc_seek =3D 0; + lp->loc_loc =3D 0; + LIST_INSERT_HEAD(&dirp->dd_td->td_locq, lp, loc_lqe= ); + } + lp =3D malloc(sizeof(struct ddloc)); if (lp =3D=3D NULL) { if (__isthreaded) I don't know if there are any downsides to special-casing zero like this, other than additional code complexity. Best, Conrad On Tue, Aug 22, 2017 at 4:38 AM, Nikolaus Rath wrote: > Hello, > > I am trying to debug a test failure of libfuse under FreeBSD. I believe > I have reduced it to the following root cause: > > Consider the following program: > > #include > #include > #include > > int main(void) { > struct dirent *e =3D NULL; > DIR *dirp; > > printf("opendir...\n"); > dirp =3D opendir("/"); > if(dirp =3D=3D NULL) { > perror("opendir"); > return 1; > } > printf("telldir: %ld\n", telldir(dirp)); > e =3D readdir(dirp); > printf("readdir: %s\n", e->d_name); > printf("telldir: %ld\n", telldir(dirp)); > e =3D readdir(dirp); > printf("readdir: %s\n", e->d_name); > printf("closedir..\n"); > closedir(dirp); > > printf("opendir...\n"); > dirp =3D opendir("/"); > if(dirp =3D=3D NULL) { > perror("opendir"); > return 1; > } > e =3D readdir(dirp); > printf("readdir: %s\n", e->d_name); > printf("telldir: %ld\n", telldir(dirp)); > e =3D readdir(dirp); > printf("readdir: %s\n", e->d_name); > printf("closedir..\n"); > closedir(dirp); > > return 0; > } > > > Under FreeBSD, running it gives: > > # ./simple > opendir... > telldir: 0 > readdir: . > telldir: 1 > readdir: .. > closedir.. > opendir... > readdir: . > telldir: 0 > readdir: .. > closedir.. > > In other words, if telldir() is called right after opendir(), it gives > an offset of zero. But if telldir() is called only after the first > readdir() call, it also gives an offset of zero. My hypothesis is that > FreeBSD actually just enumerates the different telldir() calls - is that > correct? > > Now, the offsets returned by telldir() are documented to be valid only > within a given *dirp, so FreeBSD isn't doing anything wrong. > > However, having different meanings even for an offset of zero causes > problems for libfuse, because under Linux an offset of zero is > guaranteed to mean "first entry". This is reflected in the definition of > the fuse readdir() function which always receives an *offset* parameter > that needs to have a definite value even when telldir() was never > called. If zero is suddenly also a valid telldir() return value that may > indicate some other position in the stream, things get complicated. > > Now, I think I managed to work around that by shifting all offsets by > one, but that is awkward (and I may have overlooked some problems that > the unit tests don't cover). So I am wondering: > > Is there a reason why the FreeBSD kernel could not start enumerating > telldir() offsets with 1, so that 0 can always have the same meaning? > > > Best, > -Nikolaus > > -- > GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F > > =C2=BBTime flies like an arrow, fruit flies like a Banana.= =C2=AB > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Tue Aug 22 18:30:20 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C94E3DDACBC for ; Tue, 22 Aug 2017 18:30:20 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9A8BD6FAD5 for ; Tue, 22 Aug 2017 18:30:20 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 462A720E9B for ; Tue, 22 Aug 2017 14:30:13 -0400 (EDT) Received: from frontend2 ([10.202.2.161]) by compute1.internal (MEProxy); Tue, 22 Aug 2017 14:30:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=LAlif2B6cQBhzBIDFo5ZzjClCoNSgKBI72qYaTcpwXs=; b=PRH3x3lv SIaojK57zIhMq9dO4BMAu8KRrKG9x4FeyyQhDLbFviRXHdJLN5ECgTWOvtkZDLPS D53KxUGXT8WhSqLiQVTmWTiSQZURJ9JLkBGkRtiXTmwTkdunVMo2de5SLNbePTIs ti5yfp6rYItBsRsQm0aVvl7Us3c6oXqEzDN1657/cU2C9tjILQNJuJEX7IfduSJl 2516D8ywnVPa5Pbni1m/7kBokxgwb1Q0Da2e1eTtc/lTQ2+F1bDaurv+EjODp5zq ZVcor5dnuk4sLXY3JXy7e6L2xQ5PPG/SJuuFU8r8Nr/1KQU/kAMLxW+lg8PGXefJ nrNFBcXhwdz7jg== X-ME-Sender: X-Sasl-enc: 3LJhTCu7lMxxBnnPZE33Z1cqpsK1RRzcVc2YOE/A8J4j 1503426612 Received: from ebox.rath.org (ebox.rath.org [45.79.69.51]) by mail.messagingengine.com (Postfix) with ESMTPA id EE56F2470F for ; Tue, 22 Aug 2017 14:30:12 -0400 (EDT) Received: from vostro.rath.org (vostro [192.168.12.4]) by ebox.rath.org (Postfix) with ESMTPS id 9E0DCD7 for ; Tue, 22 Aug 2017 18:30:11 +0000 (UTC) Received: by vostro.rath.org (Postfix, from userid 1000) id CF3AC1031FD; Tue, 22 Aug 2017 20:30:09 +0200 (CEST) From: Nikolaus Rath To: freebsd-fs@freebsd.org Subject: Re: Can telldir() == 0 value be made special? References: <87bmn7kf3b.fsf@vostro.rath.org> Mail-Copies-To: never Mail-Followup-To: freebsd-fs@freebsd.org Date: Tue, 22 Aug 2017 20:30:09 +0200 In-Reply-To: (Conrad Meyer's message of "Tue, 22 Aug 2017 07:42:36 -0700") Message-ID: <87lgmbfob2.fsf@vostro.rath.org> User-Agent: Gnus/5.130014 (Ma Gnus v0.14) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2017 18:30:20 -0000 On Aug 22 2017, Conrad Meyer wrote: > Hi Nikolaus, > > As you have surmised, DIR* seekpoints are created dynamically whenever > requested by user's telldir() call: Good to know, thanks! > I believe we could special case zero without breaking ABI > compatibility of correct programs. Something like this: > > --- a/lib/libc/gen/telldir.c > +++ b/lib/libc/gen/telldir.c > @@ -70,6 +70,20 @@ telldir(DIR *dirp) > } > } > if (lp =3D=3D NULL) { > + /* Create special zero telldir entry, similar to Linux */ > + if (dirp->dd_td->td_loccnt =3D=3D 0 && dirp->dd_loc !=3D = 0) { > + lp =3D malloc(sizeof(struct ddloc)); > + if (lp =3D=3D NULL) { > + if (__isthreaded) > + _pthread_mutex_unlock(&dirp->dd_l= ock); > + return (-1); > + } > + lp->loc_index =3D dirp->dd_td->td_loccnt++; > + lp->loc_seek =3D 0; > + lp->loc_loc =3D 0; > + LIST_INSERT_HEAD(&dirp->dd_td->td_locq, lp, loc_l= qe); > + } > + > lp =3D malloc(sizeof(struct ddloc)); > if (lp =3D=3D NULL) { > if (__isthreaded) > > I don't know if there are any downsides to special-casing zero like > this, other than additional code complexity. This seems a little more complex than I would have expected. Is this maybe turning zero into a valid argument for seekdir() even if telldir() has never been called? That would be even better, but the minimal change that I need is just to ensure that telldir() either (a) never returns zero, or (b) only returns zero if called at the beginning of the stream. Best, -Nikolaus --=20 GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2= =AB From owner-freebsd-fs@freebsd.org Tue Aug 22 18:35:53 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 63F2FDDB297 for ; Tue, 22 Aug 2017 18:35:53 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-yw0-x232.google.com (mail-yw0-x232.google.com [IPv6:2607:f8b0:4002:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 399706FEF5 for ; Tue, 22 Aug 2017 18:35:53 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: by mail-yw0-x232.google.com with SMTP id y64so28459968ywf.3 for ; Tue, 22 Aug 2017 11:35:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-transfer-encoding; bh=z79oEXAr/WwBxxwPLYlMV0rS32/Tmc+3lRHjRN/ixC0=; b=O1YrxeUQ8dE6Vj5bcR3lx+VBNdwjd5gwchsU0GAktSA+iTFRyn0zgVxD2tnT75b5Ab BnP5q3yLgDeE5hephVJCWJ39HLHl/Hv/2DcCocpSLY20qLJVGdURvFz71HudRmCX0zjS U0dPfV5+HggdkOfcMgVj4lcSXQ7GHVKaWI6p8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-transfer-encoding; bh=z79oEXAr/WwBxxwPLYlMV0rS32/Tmc+3lRHjRN/ixC0=; b=M7GHAxE1Yp407SQP8f52Y/gBV6r8OfV/ToftDKfgJ7dab4TUnRfvf1iex7dpJNfZRO zxYDFivOXBVwzUAgk3BARckY+eL6F4fLEeSozVh8xUaa7/S8QhWjaDNd4hWOQzDWyI1Y fznHXHroEhgU+xIUQnPcnNv9Q42wT8lpbllRqU/CKHkHTRMT8Aj27lx6gqU/iLwkET8H UecnukRTANZSeqKcBBjN4it2BD5QUCkDzy3cazNbVeawIZre927jQFEn9dTw1grwpzj5 a4VVRFcOlOaR90CxZ3+PUqTBWo4NGOkbrVkEJViImyK1W0f2+DAq+7M8yHsJ42+M+rHG XtkA== X-Gm-Message-State: AHYfb5jB4P5vN0dcTsmWeyoxzf0kQpMwveR7/ab9qkSaiSsiKr/XiFQe /Sw9rjzvXvm37Tce0cb45OA2HUaG0UvW X-Received: by 10.13.230.74 with SMTP id p71mr57511ywe.456.1503426952053; Tue, 22 Aug 2017 11:35:52 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.57.21 with HTTP; Tue, 22 Aug 2017 11:35:21 -0700 (PDT) In-Reply-To: <87lgmbfob2.fsf@vostro.rath.org> References: <87bmn7kf3b.fsf@vostro.rath.org> <87lgmbfob2.fsf@vostro.rath.org> From: Eitan Adler Date: Tue, 22 Aug 2017 14:35:21 -0400 Message-ID: Subject: Re: Can telldir() == 0 value be made special? To: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2017 18:35:53 -0000 On 22 August 2017 at 14:30, Nikolaus Rath wrote: > On Aug 22 2017, Conrad Meyer wrote: >> Hi Nikolaus, >> >> As you have surmised, DIR* seekpoints are created dynamically whenever >> requested by user's telldir() call: > > Good to know, thanks! > >> I believe we could special case zero without breaking ABI >> compatibility of correct programs. Something like this: >> >> --- a/lib/libc/gen/telldir.c >> +++ b/lib/libc/gen/telldir.c >> @@ -70,6 +70,20 @@ telldir(DIR *dirp) >> } >> } >> if (lp =3D=3D NULL) { >> + /* Create special zero telldir entry, similar to Linux *= / >> + if (dirp->dd_td->td_loccnt =3D=3D 0 && dirp->dd_loc !=3D= 0) { >> + lp =3D malloc(sizeof(struct ddloc)); >> + if (lp =3D=3D NULL) { >> + if (__isthreaded) >> + _pthread_mutex_unlock(&dirp->dd_= lock); >> + return (-1); >> + } >> + lp->loc_index =3D dirp->dd_td->td_loccnt++; >> + lp->loc_seek =3D 0; >> + lp->loc_loc =3D 0; >> + LIST_INSERT_HEAD(&dirp->dd_td->td_locq, lp, loc_= lqe); >> + } >> + >> lp =3D malloc(sizeof(struct ddloc)); >> if (lp =3D=3D NULL) { >> if (__isthreaded) >> >> I don't know if there are any downsides to special-casing zero like >> this, other than additional code complexity. > > This seems a little more complex than I would have expected. > Is this > maybe turning zero into a valid argument for seekdir() even if telldir() > has never been called? No, this code is called inside of telldir. I'd be against adding that behavior by contract since it complicates the contract of seekdir. No objection to this behavior by implementation though if it is simpler than modifying telldir (doubtful). What this is doing is ensuring that there will always be a 0 entry *after* telldir is called, and that seekdir(0) will return to the start of the directory if called with an argument of 0. > That would be even better, but the minimal change that I need is just to > ensure that telldir() either (a) never returns zero, or (b) only returns > zero if called at the beginning of the stream. > > > Best, > -Nikolaus > > -- > GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F > > =C2=BBTime flies like an arrow, fruit flies like a Banana.= =C2=AB > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --=20 Eitan Adler From owner-freebsd-fs@freebsd.org Tue Aug 22 19:28:32 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 236D6DDE77A for ; Tue, 22 Aug 2017 19:28:32 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-oi0-f43.google.com (mail-oi0-f43.google.com [209.85.218.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E419771BA0 for ; Tue, 22 Aug 2017 19:28:31 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-oi0-f43.google.com with SMTP id j144so68137003oib.1 for ; Tue, 22 Aug 2017 12:28:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to; bh=IImXRiawThY9/iH8Hz20LkYtc5x8446sg5IFl6BvjjI=; b=sLw9JtLfFDNyDcHr0No0+yldMNpT2O+vjNoRdu9yR2/gZKYVsL7kUSY/rwuDQsnRQB nspNC3basoIumw4y4goah3yBgePex5Xwp3mhH70tkd8X9Nqpmwr4CWWgOP0tTWFxDM+1 YVrHwjERtDRvpYUUTdQOi5GHa5CjwrYqY1lOWkhpJChEC+DWLqyyn5I1YgYL0BJm1rBs 8lhZMGXJ/GCHpFcmhvOzJjW1b8tNWdCFsusDnLB6LBaj4PluFkFUuab7/rw2wUyUL/dw jNx1jHqn7RZdGJDBQqTEyuNhurw8qkVsP60pqrrVsUg/i9ULxmPvOoDsOqG91M9V6L41 aVHQ== X-Gm-Message-State: AHYfb5iNoFbyENH7WrNWOJlzzrrqaCba6n3A2hbTI8hinV0XsfG4FRt9 +qZd8dL8bZJ3avF7H44= X-Received: by 10.202.196.213 with SMTP id u204mr214495oif.318.1503428730626; Tue, 22 Aug 2017 12:05:30 -0700 (PDT) Received: from mail-it0-f46.google.com (mail-it0-f46.google.com. [209.85.214.46]) by smtp.gmail.com with ESMTPSA id s62sm127421oie.16.2017.08.22.12.05.30 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 Aug 2017 12:05:30 -0700 (PDT) Received: by mail-it0-f46.google.com with SMTP id o19so287768ito.0 for ; Tue, 22 Aug 2017 12:05:30 -0700 (PDT) X-Received: by 10.36.129.138 with SMTP id q132mr838238itd.174.1503428729893; Tue, 22 Aug 2017 12:05:29 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.2.81.131 with HTTP; Tue, 22 Aug 2017 12:05:29 -0700 (PDT) In-Reply-To: References: <87bmn7kf3b.fsf@vostro.rath.org> <87lgmbfob2.fsf@vostro.rath.org> From: Conrad Meyer Date: Tue, 22 Aug 2017 12:05:29 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Can telldir() == 0 value be made special? To: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2017 19:28:32 -0000 One could also make it reserve the zero slot for the loc == 0 entry, but not create it dynamically. Just set loccnt=1 at initialization and then special-case dd_loc==0. Like this: --- a/lib/libc/gen/opendir.c +++ b/lib/libc/gen/opendir.c @@ -295,7 +295,7 @@ __opendir_common(int fd, int flags, bool use_current_pos) dirp->dd_lock = NULL; dirp->dd_td = (struct _telldir *)((char *)dirp + sizeof(DIR)); LIST_INIT(&dirp->dd_td->td_locq); - dirp->dd_td->td_loccnt = 0; + dirp->dd_td->td_loccnt = 1; dirp->dd_compat_de = NULL; /* --- a/lib/libc/gen/telldir.c +++ b/lib/libc/gen/telldir.c @@ -76,7 +76,10 @@ telldir(DIR *dirp) _pthread_mutex_unlock(&dirp->dd_lock); return (-1); } - lp->loc_index = dirp->dd_td->td_loccnt++; + if (dirp->dd_loc == 0) + lp->loc_index = 0; + else + lp->loc_index = dirp->dd_td->td_loccnt++; lp->loc_seek = dirp->dd_seek; lp->loc_loc = dirp->dd_loc; if (flp != NULL) Best, Conrad On Tue, Aug 22, 2017 at 11:35 AM, Eitan Adler wrote: > On 22 August 2017 at 14:30, Nikolaus Rath wrote: >> On Aug 22 2017, Conrad Meyer wrote: >>> Hi Nikolaus, >>> >>> As you have surmised, DIR* seekpoints are created dynamically whenever >>> requested by user's telldir() call: >> >> Good to know, thanks! >> >>> I believe we could special case zero without breaking ABI >>> compatibility of correct programs. Something like this: >>> >>> --- a/lib/libc/gen/telldir.c >>> +++ b/lib/libc/gen/telldir.c >>> @@ -70,6 +70,20 @@ telldir(DIR *dirp) >>> } >>> } >>> if (lp == NULL) { >>> + /* Create special zero telldir entry, similar to Linux */ >>> + if (dirp->dd_td->td_loccnt == 0 && dirp->dd_loc != 0) { >>> + lp = malloc(sizeof(struct ddloc)); >>> + if (lp == NULL) { >>> + if (__isthreaded) >>> + _pthread_mutex_unlock(&dirp->dd_lock); >>> + return (-1); >>> + } >>> + lp->loc_index = dirp->dd_td->td_loccnt++; >>> + lp->loc_seek = 0; >>> + lp->loc_loc = 0; >>> + LIST_INSERT_HEAD(&dirp->dd_td->td_locq, lp, loc_lqe); >>> + } >>> + >>> lp = malloc(sizeof(struct ddloc)); >>> if (lp == NULL) { >>> if (__isthreaded) >>> >>> I don't know if there are any downsides to special-casing zero like >>> this, other than additional code complexity. >> >> This seems a little more complex than I would have expected. > >> Is this >> maybe turning zero into a valid argument for seekdir() even if telldir() >> has never been called? > > No, this code is called inside of telldir. I'd be against adding that > behavior by contract since it complicates the contract of seekdir. No > objection to this behavior by implementation though if it is simpler > than modifying telldir (doubtful). > > What this is doing is ensuring that there will always be a 0 entry > *after* telldir is called, and that seekdir(0) will return to the > start of the directory if called with an argument of 0. > > >> That would be even better, but the minimal change that I need is just to >> ensure that telldir() either (a) never returns zero, or (b) only returns >> zero if called at the beginning of the stream. From owner-freebsd-fs@freebsd.org Tue Aug 22 19:51:15 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36ABDDDFDC1 for ; Tue, 22 Aug 2017 19:51:15 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670042.outbound.protection.outlook.com [40.107.67.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D8C66729CB for ; Tue, 22 Aug 2017 19:51:13 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0494.CANPRD01.PROD.OUTLOOK.COM (10.165.220.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1362.18; Tue, 22 Aug 2017 19:51:11 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.1362.019; Tue, 22 Aug 2017 19:51:11 +0000 From: Rick Macklem To: Ronald Klop , "freebsd-fs@freebsd.org" Subject: Re: when has a pNFS data server failed? Thread-Topic: when has a pNFS data server failed? Thread-Index: AQHTGGwqSs0WOO4N5Uy/bwL72vYOsKKQHCeAgACtylg= Date: Tue, 22 Aug 2017 19:51:11 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YTXPR01MB0494; 6:WIRttAOU//CUiQej5C3pGggM7EcS6kNv5OZIXtrWuZuXvKpYnZAur+klHeqBCqhaxeOEbjR5nhoJ8fTsIOoeUv2FVZRmmPYCkaQWXIf031TVnMgKJJSvN2PxwcygDDfezAXdz0d3Jx50ZbeFxgxWP/6LjETOaRMbyli5SB5++pZMbc6jyhhr5N1a9JmvC4PFtj+t0JF2ZbJXr5azxPTuKj/lhtJH+wz6klZm/Yw5T+FHtmOmsbi+EsbAONVqUW6FPzQCRsTf4EJqO46MV78d/4ubF/r8ELFLFGKS6LOrgXEyZU3oqOvCaFZHaT3OA41vcbfFy/VIBJ7tlGZTXDGBuQ==; 5:evsXEPspRFx5cwF8vRlZn2OnyXVDKxBFJajbXbLHg6al6WCvn5LdsxCDWWfkk+UUUuVUi0JFkty/Aelsu8bxZJE3S2osXnnAhVKMymoVKuCSok0b0rBLyPv7O63fCTyNw24pEVNtoDIeP9G6iHi11A==; 24:EH/ayTPgGu2U1QiTB0AWCessOi+7swtXlkoZ4ZQI4HJQuiVix5cQoLFbVGzn67V/DINWK46VlTam3gudkScXo2y6KYj+QjFGlco5lX+zsaQ=; 7:TlFxmXeO/GU6GGmglMfi3yy/+pQnfb6NgDu5E/6hdZ93PlWfKc1wDYaubp2OZvEMHpueMOuofm58Pj4vvjk0x2m1NbxOPDz1z5mfW+lrQG6vOu8Q9LYGEyo2NVsuKIz5mIrPwUd/mcNB+HcLBJeIauLvkMvrQVm5HLlCU2u3vAuO6EeWSPDy1QZBQJjQPbwmFmktDdslGRUfHZz4PBu58AL2KHw3Algm9WeMxMA4kDU= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 09174523-4444-4e9c-3ca6-08d4e99723d9 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603173)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:YTXPR01MB0494; x-ms-traffictypediagnostic: YTXPR01MB0494: x-exchange-antispam-report-test: UriScan:(158342451672863); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93001095)(6041248)(20161123564025)(20161123562025)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:YTXPR01MB0494; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:YTXPR01MB0494; x-forefront-prvs: 04073E895A x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(189002)(199003)(51914003)(24454002)(2906002)(77096006)(102836003)(2501003)(6306002)(53936002)(9686003)(55016002)(68736007)(3660700001)(74316002)(305945005)(6506006)(6436002)(97736004)(3280700002)(81156014)(76176999)(50986999)(14454004)(8936002)(25786009)(189998001)(54356999)(229853002)(8676002)(81166006)(2950100002)(33656002)(86362001)(478600001)(966005)(101416001)(74482002)(2900100001)(105586002)(106356001)(7696004)(5660300001)(6246003); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0494; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Aug 2017 19:51:11.6230 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0494 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2017 19:51:15 -0000 Ronald Klop wrote: >On Fri, 18 Aug 2017 23:52:12 +0200, Rick Macklem >wrote: >> This is kind of a "big picture" question that I thought I 'd throw out. >> >> As a brief background, I now have the code for running mirrored pNFS >> Data Servers >> working for normal operation. You can look at: >> http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt >> if you are interested in details related to the pNFS server code/testing= . >> >> So, now I am facing the interesting part: >> 1 - The Metadata Server (MDS) needs to decide that a mirrored DS has >> failed at some >> point. Once that happens, it stops using the DS, etc. >> --> This brings me to the question of "when should the MDS decide that >> the DS has >> failed and should be taken offline?". >> - I'm not up to date w.r.t. the TCP stack, so I'm not sure how >> long it will take for the >> TCP connection to decide that a DS server is no longer working >> and fail the TCP >> connection. I think it takes a fair amount of time, so I'm not >> sure if TCP connection >> loss is a good indicator of DS server failure or not? >> - It seems to me that the MDS should wait a fairly long time before >> failing the DS, >> since this will have a major impact on the pNFS server, requiring >> repair/resilvering >> by a sysadmin once it happens. >> So, any comments or thoughts on this? rick > >This is a quite common problem for all clustered/connected systems. I >think there is no general answer. And there are a lot of papers written >about it. If you have a suggestion for one good paper, I might be willing to read it. Short answer is I'm retired after 30years of working for a University and I= have roughly a 0 interest in reading academic papers. >For example: in NFS you have the 'soft' option. It is recommended not to >use it. I can imagine that if your home-dir or /usr is mounted over NFS, >but at work I want my http-servers to not hang and just give an IO-error >when the backend fileserver with data is gone. >Something similar happens here. Yes. However, the analogy only works so far, in that a failure of a "soft" = mount affects integrity of the file, if it is a write that fails. In this case, there shouldn't be data corruption/loss, however there may be degraded performance during the mirror failure and subsequent resilvering. (A closer analogy might be a drive failure when in a mirrored configuration with another drive. These days drive hardware does try to indicate "hardwa= re health", which the mirrored server may not provide, at least in the early version.) > Doesn't the protocol definition say something about this? Nope, except for some "on the wire" information that the pNFS client can pr= ovide to indicate to the MDS that it is having problems with a DS. (The RFCs deal with what goes on the wire and not how servers get implement= ed.) > Or what do other implementations do? I have no idea. At this point, all extant pNFS server implementations are p= roprietary blobs, such as a Netapp clustered configuration. I've only seen "high level= " white papers (one notch away from marketing). To be honest, I think the answer for version 1 will come down to... How long should the MDS try to communicate with the DS before it gives up a= nd considers it failed? It will probably be setable via a sysctl, but does need a reasonable defaul= t value. (A "very large" value would indicate "leave it for the sysadmin to decide a= nd do manually.) I also think there might be certain error returns from sosend()/sorecieve()= that may want special handling. A simple example I experienced in recent testing was... - One system was misconfigured with the same IP# as one of the DS systems. After fixing the misconfiguration, the pNFS server was wedged because it= had a bogus arp entry so it couldn't talk to the one mirror. --> This was easily handled by a "arp -d" done by me on the MDS, but if the= MDS had given up on the DS before I did that, it would have been a lot mo= re work to fix. (The bogus arp entry had a very long timeout on it.) Anyhow, thanks for the comments and we'll see if others have comments, rick From owner-freebsd-fs@freebsd.org Wed Aug 23 04:16:48 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7892BDD974C for ; Wed, 23 Aug 2017 04:16:48 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-246-50.A444.priv.bahnhof.se [155.4.246.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 139A614DF for ; Wed, 23 Aug 2017 04:16:47 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 7754748A15; Wed, 23 Aug 2017 06:10:08 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id a_lWcLu7T4SY; Wed, 23 Aug 2017 06:10:06 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 310E648A1A; Wed, 23 Aug 2017 06:10:06 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se 310E648A1A X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Lj8dK9A8UWdP; Wed, 23 Aug 2017 06:10:05 +0200 (CEST) Received: from mail.inparadise.se (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id BE92648A15; Wed, 23 Aug 2017 06:10:05 +0200 (CEST) Date: Wed, 23 Aug 2017 06:10:03 +0200 (CEST) Subject: Re: when has a pNFS data server failed? Message-ID: <2fbb5be6-f9c0-467a-a200-1783cf2c4a67@email.android.com> X-Android-Message-ID: <2fbb5be6-f9c0-467a-a200-1783cf2c4a67@email.android.com> To: Rick Macklem Cc: Ronald Klop , freebsd-fs@freebsd.org Importance: Normal X-Priority: 3 X-MSMail-Priority: Normal From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= X-Originating-IP: [172.16.1.122, 127.0.0.1] X-Mailer: Zimbra 8.7.1_GA_1670 (Android-Mail/7.7.16.164685024.release(...883836) devip=172.16.1.122 ZPZB/66) Thread-Index: /NjliFtpbMti1GadJaeNM8fbRC60xw== Thread-Topic: when has a pNFS data server failed? MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 04:16:48 -0000 From owner-freebsd-fs@freebsd.org Wed Aug 23 10:11:41 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30896DE2F98 for ; Wed, 23 Aug 2017 10:11:41 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 01AC769DB3 for ; Wed, 23 Aug 2017 10:11:40 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 8FA3E21B08 for ; Wed, 23 Aug 2017 06:11:38 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Wed, 23 Aug 2017 06:11:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=jEcOKpNm6z9a8pmY79UnsJNWXCrPT17jeWjkqEhtZVw=; b=OQtf7jyR incj9tn8Ng2SdkzIb/HuT8UYBBMz4NK//cQ3Z4o5EXCiaABJw4ReCF7+VsnNyVc8 IQV0ou4z2N9MbpGDIEeNU+WXZWHGLBT5Jem+FsnFxnSTrU2wWzZDuXz3E2mT+zUK WKPk8ZXfGub1re2VMIjPUv9vhj8O9IpSEt+lfaUzZnSdQwH7vI2saKduS7yfub7i EEwnZeCj+3cG4ZFjiN5RuqitG/QvVJyNZFD5ekmCo/gosh+s40OZBU8iN8uR7Rgr sfc0953C8R1LktPEf9Nc4zTiSz6v6pzeygnyokswwFUDktYxLG0JWtP/XOjEC6gT xxKUKFRW40jx8Q== X-ME-Sender: X-Sasl-enc: 88x/h//SfweNYgVSMenFgloqsd0uF9cr2Ho4xRj+1Gey 1503483098 Received: from ebox.rath.org (ebox.rath.org [45.79.69.51]) by mail.messagingengine.com (Postfix) with ESMTPA id 4B9347FA1C for ; Wed, 23 Aug 2017 06:11:38 -0400 (EDT) Received: from vostro.rath.org (vostro [192.168.12.4]) by ebox.rath.org (Postfix) with ESMTPS id 48E92116 for ; Wed, 23 Aug 2017 10:11:37 +0000 (UTC) Received: by vostro.rath.org (Postfix, from userid 1000) id A88091031FE; Wed, 23 Aug 2017 12:11:35 +0200 (CEST) From: Nikolaus Rath To: freebsd-fs@freebsd.org Subject: Re: Can telldir() == 0 value be made special? References: <87bmn7kf3b.fsf@vostro.rath.org> <87lgmbfob2.fsf@vostro.rath.org> Mail-Copies-To: never Mail-Followup-To: freebsd-fs@freebsd.org Date: Wed, 23 Aug 2017 12:11:35 +0200 In-Reply-To: (Conrad Meyer's message of "Tue, 22 Aug 2017 12:05:29 -0700") Message-ID: <87ziaqr3u0.fsf@vostro.rath.org> User-Agent: Gnus/5.130014 (Ma Gnus v0.14) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 10:11:41 -0000 On Aug 22 2017, Conrad Meyer wrote: > One could also make it reserve the zero slot for the loc =3D=3D 0 entry, > but not create it dynamically. Just set loccnt=3D1 at initialization > and then special-case dd_loc=3D=3D0. Like this: > > --- a/lib/libc/gen/opendir.c > +++ b/lib/libc/gen/opendir.c > @@ -295,7 +295,7 @@ __opendir_common(int fd, int flags, bool use_current_= pos) > dirp->dd_lock =3D NULL; > dirp->dd_td =3D (struct _telldir *)((char *)dirp + sizeof(DIR)); > LIST_INIT(&dirp->dd_td->td_locq); > - dirp->dd_td->td_loccnt =3D 0; > + dirp->dd_td->td_loccnt =3D 1; > dirp->dd_compat_de =3D NULL; > > /* > --- a/lib/libc/gen/telldir.c > +++ b/lib/libc/gen/telldir.c > @@ -76,7 +76,10 @@ telldir(DIR *dirp) > _pthread_mutex_unlock(&dirp->dd_lock); > return (-1); > } > - lp->loc_index =3D dirp->dd_td->td_loccnt++; > + if (dirp->dd_loc =3D=3D 0) > + lp->loc_index =3D 0; > + else > + lp->loc_index =3D dirp->dd_td->td_loccnt++; > lp->loc_seek =3D dirp->dd_seek; > lp->loc_loc =3D dirp->dd_loc; > if (flp !=3D NULL) > This would improve Linux/fuse compatibility a lot, and seems like a minimal increase in complexity. Is there any way I can help to get it merged? Best, -Nikolaus --=20 GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2= =AB From owner-freebsd-fs@freebsd.org Wed Aug 23 10:41:22 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E0FCDE3818 for ; Wed, 23 Aug 2017 10:41:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 091036A94F for ; Wed, 23 Aug 2017 10:41:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v7NAfL7x027097 for ; Wed, 23 Aug 2017 10:41:21 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 221707] zfs receive dry run fails but backup works Date: Wed, 23 Aug 2017 10:41:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 10:41:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221707 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Aug 23 12:06:35 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBC7BDE57BE for ; Wed, 23 Aug 2017 12:06:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CA84B6D486 for ; Wed, 23 Aug 2017 12:06:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v7NC6Z5v060856 for ; Wed, 23 Aug 2017 12:06:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 221707] zfs receive dry run fails but backup works Date: Wed, 23 Aug 2017 12:06:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 12:06:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221707 --- Comment #1 from Andriy Gapon --- I am not sure that zfs can simulate reception of replications streams sufficiently well. It lacks the machinery to simulate creation of new data= sets which might be required to simulate reception of subordinate datasets. But= it does detect when a dataset that is supposed to be received already exists. I am not sure if it is worth the trouble perfecting the dry mode. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Aug 23 12:09:49 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B726DE58EC for ; Wed, 23 Aug 2017 12:09:49 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [46.17.166.21]) by mx1.freebsd.org (Postfix) with ESMTP id E25966D637 for ; Wed, 23 Aug 2017 12:09:47 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (higson.cam.lispworks.com [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.15.2/8.15.2) with ESMTP id v7NC9c8D075146; Wed, 23 Aug 2017 13:09:38 +0100 (BST) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (localhost.localdomain [127.0.0.1]) by higson.cam.lispworks.com (8.14.4) id v7NC9cSc008397; Wed, 23 Aug 2017 13:09:38 +0100 Received: (from martin@localhost) by higson.cam.lispworks.com (8.14.4/8.14.4/Submit) id v7NC9cYk008393; Wed, 23 Aug 2017 13:09:38 +0100 Date: Wed, 23 Aug 2017 13:09:38 +0100 Message-Id: <201708231209.v7NC9cYk008393@higson.cam.lispworks.com> From: Martin Simmons To: freebsd-fs@freebsd.org In-reply-to: (message from Conrad Meyer on Tue, 22 Aug 2017 12:05:29 -0700) Subject: Re: Can telldir() == 0 value be made special? References: <87bmn7kf3b.fsf@vostro.rath.org> <87lgmbfob2.fsf@vostro.rath.org> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 12:09:49 -0000 Does this solution (and the previous one) need to check for dd_seek == 0 as well? It looks like dd_loc == 0 is also true after readdir has returned NULL for eof. __Martin >>>>> On Tue, 22 Aug 2017 12:05:29 -0700, Conrad Meyer said: > > One could also make it reserve the zero slot for the loc == 0 entry, > but not create it dynamically. Just set loccnt=1 at initialization > and then special-case dd_loc==0. Like this: > > --- a/lib/libc/gen/opendir.c > +++ b/lib/libc/gen/opendir.c > @@ -295,7 +295,7 @@ __opendir_common(int fd, int flags, bool use_current_pos) > dirp->dd_lock = NULL; > dirp->dd_td = (struct _telldir *)((char *)dirp + sizeof(DIR)); > LIST_INIT(&dirp->dd_td->td_locq); > - dirp->dd_td->td_loccnt = 0; > + dirp->dd_td->td_loccnt = 1; > dirp->dd_compat_de = NULL; > > /* > --- a/lib/libc/gen/telldir.c > +++ b/lib/libc/gen/telldir.c > @@ -76,7 +76,10 @@ telldir(DIR *dirp) > _pthread_mutex_unlock(&dirp->dd_lock); > return (-1); > } > - lp->loc_index = dirp->dd_td->td_loccnt++; > + if (dirp->dd_loc == 0) > + lp->loc_index = 0; > + else > + lp->loc_index = dirp->dd_td->td_loccnt++; > lp->loc_seek = dirp->dd_seek; > lp->loc_loc = dirp->dd_loc; > if (flp != NULL) > > Best, > Conrad > > > On Tue, Aug 22, 2017 at 11:35 AM, Eitan Adler wrote: > > On 22 August 2017 at 14:30, Nikolaus Rath wrote: > >> On Aug 22 2017, Conrad Meyer wrote: > >>> Hi Nikolaus, > >>> > >>> As you have surmised, DIR* seekpoints are created dynamically whenever > >>> requested by user's telldir() call: > >> > >> Good to know, thanks! > >> > >>> I believe we could special case zero without breaking ABI > >>> compatibility of correct programs. Something like this: > >>> > >>> --- a/lib/libc/gen/telldir.c > >>> +++ b/lib/libc/gen/telldir.c > >>> @@ -70,6 +70,20 @@ telldir(DIR *dirp) > >>> } > >>> } > >>> if (lp == NULL) { > >>> + /* Create special zero telldir entry, similar to Linux */ > >>> + if (dirp->dd_td->td_loccnt == 0 && dirp->dd_loc != 0) { > >>> + lp = malloc(sizeof(struct ddloc)); > >>> + if (lp == NULL) { > >>> + if (__isthreaded) > >>> + _pthread_mutex_unlock(&dirp->dd_lock); > >>> + return (-1); > >>> + } > >>> + lp->loc_index = dirp->dd_td->td_loccnt++; > >>> + lp->loc_seek = 0; > >>> + lp->loc_loc = 0; > >>> + LIST_INSERT_HEAD(&dirp->dd_td->td_locq, lp, loc_lqe); > >>> + } > >>> + > >>> lp = malloc(sizeof(struct ddloc)); > >>> if (lp == NULL) { > >>> if (__isthreaded) > >>> > >>> I don't know if there are any downsides to special-casing zero like > >>> this, other than additional code complexity. > >> > >> This seems a little more complex than I would have expected. > > > >> Is this > >> maybe turning zero into a valid argument for seekdir() even if telldir() > >> has never been called? > > > > No, this code is called inside of telldir. I'd be against adding that > > behavior by contract since it complicates the contract of seekdir. No > > objection to this behavior by implementation though if it is simpler > > than modifying telldir (doubtful). > > > > What this is doing is ensuring that there will always be a 0 entry > > *after* telldir is called, and that seekdir(0) will return to the > > start of the directory if called with an argument of 0. > > > > > >> That would be even better, but the minimal change that I need is just to > >> ensure that telldir() either (a) never returns zero, or (b) only returns > >> zero if called at the beginning of the stream. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Wed Aug 23 12:36:08 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A2B9DE6466 for ; Wed, 23 Aug 2017 12:36:08 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660042.outbound.protection.outlook.com [40.107.66.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B05026E466 for ; Wed, 23 Aug 2017 12:36:06 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0702.CANPRD01.PROD.OUTLOOK.COM (10.165.221.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1362.18; Wed, 23 Aug 2017 12:36:04 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.1362.019; Wed, 23 Aug 2017 12:36:04 +0000 From: Rick Macklem To: =?iso-8859-1?Q?Karli_Sj=F6berg?= CC: Ronald Klop , "freebsd-fs@freebsd.org" Subject: Re: when has a pNFS data server failed? Thread-Topic: when has a pNFS data server failed? Thread-Index: /NjliFtpSs0WOO4N5Uy/bwL72vYOsPRtBMq3 Date: Wed, 23 Aug 2017 12:36:04 +0000 Message-ID: References: <2fbb5be6-f9c0-467a-a200-1783cf2c4a67@email.android.com> In-Reply-To: <2fbb5be6-f9c0-467a-a200-1783cf2c4a67@email.android.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YTXPR01MB0702; 6:mlwTYuQQPkWrkpbGfjNOhp/tNhgugqt6ClUXty01O1bPM9Q1X5MvxQBlk4S+clPWdUqWGNLfOxXiwY0G86Gj0tlZpl7/ZTM1f1ZxKJNQIVU40GYlTtWv9qfdWvG0mW6NUDSgsARGnRJ6C4uFGn7cQ4/bOtBpylpicyprbSdgPGBVBCUPOVGliC+G1X6z0wnIozFLtV4Qhkuh48tq8npKvmtjKYKFbXSGtut6BGLCxL0RLJZVZ4HKhJgPiVI0utRc+uYGdZY6Z2ZuEW/pqEK/W2j72/FbJcKCUh5LppoTXOHyxK0zvu48jO+63b4DtOGqjrlBIbNWFNBKt439Ms7S0Q==; 5:AHeINJVxcZ6AaJ2Tfe6+aCkfeE8a+z0PnG3ACa96iXWOBtuRbHRSe0wDiUJo00xX1cQ/5lZgKMIU0zBAbnsgaDzPwpuW6Sk2PSt+Htw79gmyPRMzX58HbCKrGlscoW/xk8EHKVGKLeiZBzNqDjIiqA==; 24:u2dYR2y2EhnnCy5QyXDQ59AagQP8KXihaee6s1chJH+hk8V3dkqFyhTG7eyjExafVTmqBY0pri1Q/kjBVYtuTqL3GarpZJSGK/t1HKQ1xdg=; 7:oyb1t8VJXf+WJqevWoybLuqfqm05/DuN2187SyDtwJ0XOLI+Sw9PlrM0+VyfD2uoF/y3n0HhimQyKR7NYcHPq1MsbAJwD4ogqf8Lr/IK58+ZiGdQ1izxRxpobmSs3pMtN4d/kDpHUmNHkqZUzE7sBdVG7BCGoREBCzuDFiK9oB7FVSDmER0Ew6WORrw9ElYJqyJGCOVD2xyWDJ3tj4Ur67ChaChhb0t1ulMnkuZYxqs= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: a41835e2-5dd2-4ac0-6572-08d4ea238504 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603031)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:YTXPR01MB0702; x-ms-traffictypediagnostic: YTXPR01MB0702: x-exchange-antispam-report-test: UriScan:(61668805478150)(158342451672863)(82924173822182)(245836752223355); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(100000703101)(100105400095)(3002001)(6041248)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123564025)(20161123555025)(20161123562025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:YTXPR01MB0702; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:YTXPR01MB0702; x-forefront-prvs: 040866B734 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(24454002)(189002)(199003)(51914003)(5660300001)(8936002)(105586002)(7696004)(97736004)(25786009)(33656002)(6246003)(6916009)(106356001)(81166006)(14454004)(54356999)(81156014)(9686003)(50986999)(4326008)(8676002)(74316002)(76176999)(101416001)(2950100002)(5890100001)(189998001)(2906002)(77096006)(305945005)(102836003)(55016002)(68736007)(6506006)(53936002)(54906002)(6436002)(478600001)(86362001)(6306002)(229853002)(3280700002)(2900100001)(110136004)(74482002)(3660700001); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0702; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Aug 2017 12:36:04.1401 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0702 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 12:36:08 -0000 Karli Sj=F6berg wrote: [stuff snipped for brevity] >>Rick Macklem wrote: >>To be honest, I think the answer for version 1 will come down to... >> >>How long should the MDS try to communicate with the DS before it gives up= and >>considers it failed? >> >>It will probably be setable via a sysctl, but does need a reasonable defa= ult value. >>(A "very large" value would indicate "leave it for the sysadmin to decide= and do >>manually.) [more stuff snipped] >This is what one prominent "customer" says about timeout: >https://kb.vmware.com/selfservice/microsites/search.do?language=3Den_US&cm= d=3DdisplayKC&externalId=3D1009465 >"These issues occur when the guest operating system timeout values are exc= eeded for >attached storage disks. This may be caused by an underlying stor= age problem or due to >brief transient pauses during normal operations (suc= h as path failover). To accommodate >transient events, the VMware Tools inc= reases the SCSI disk timeout to 60 seconds for >Virtual Infrastructure 3 an= d 180 seconds for vSphere 4 and higher." > >Which means that you have a minute before the "customers" start complainin= g:) Thanks. I was thinking that a minute or two is about what the default might= want to be. It may need to be longer than that, since a DS needs to be able to r= eboot and start servicing RPCs before this timeout happens as one example. (Fortunately a DS does not need to wait for the "grace period that an NFSv4= /MDS server does after boot, since that time is for clients to recover locks an= d the locks are handled by the MDS and not the DSs.) Thanks for the comment, rick From owner-freebsd-fs@freebsd.org Wed Aug 23 12:38:11 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F869DE6544 for ; Wed, 23 Aug 2017 12:38:11 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [46.17.166.21]) by mx1.freebsd.org (Postfix) with ESMTP id D04036E57D for ; Wed, 23 Aug 2017 12:38:09 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (higson.cam.lispworks.com [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.15.2/8.15.2) with ESMTP id v7NCc4XH076716; Wed, 23 Aug 2017 13:38:04 +0100 (BST) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (localhost.localdomain [127.0.0.1]) by higson.cam.lispworks.com (8.14.4) id v7NCc4QM008612; Wed, 23 Aug 2017 13:38:04 +0100 Received: (from martin@localhost) by higson.cam.lispworks.com (8.14.4/8.14.4/Submit) id v7NCc4f7008608; Wed, 23 Aug 2017 13:38:04 +0100 Date: Wed, 23 Aug 2017 13:38:04 +0100 Message-Id: <201708231238.v7NCc4f7008608@higson.cam.lispworks.com> From: Martin Simmons To: Larry Rosenman CC: freebsd-fs@FreeBSD.org In-reply-to: <20170822141419.v6ely424bjvcnzoj@lrosenman.local> (message from Larry Rosenman on Tue, 22 Aug 2017 09:14:19 -0500) Subject: Re: ENAMETOOLONG? References: <20170822141419.v6ely424bjvcnzoj@lrosenman.local> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 12:38:11 -0000 >>>>> On Tue, 22 Aug 2017 09:14:19 -0500, Larry Rosenman said: > > Are there plans to increase the max length for zfs? > > [1239928] ZFS WARNING: Unable to create ZVOL zroot/vm-bhyve/freebsd11/disk0@zfs-auto-snap_hourly-2017-08-22-09h00 (error=63). I think this is hitting the limit on SPECNAMELEN when it tries to create the device node. __Martin From owner-freebsd-fs@freebsd.org Wed Aug 23 14:15:12 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA085DE8167 for ; Wed, 23 Aug 2017 14:15:12 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (unknown [IPv6:2001:470:1f0f:3ad:bb:dcff:fe50:d900]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A964071FD6 for ; Wed, 23 Aug 2017 14:15:11 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Content-transfer-encoding:Content-type:Mime-version:In-Reply-To: References:Message-ID:CC:To:From:Subject:Date:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=K2nxwFvNx+FTI2NN4kPFqt2hyGqrqJNkswWRQevdwk0=; b=PRWQaB07BLcIxEFn8jNqeaBL7a JUsFCFEHqMj8nVciv608UBSVLs49DjDlRni9pqcBgk3KmXdwpnZMh6vu5tuY6oL5Y6y1A2IOGKC1A 7L1FDib9VCySLX94HeDlZ42yAbWjRz2FPR937Do9a5bju1dKS6Ysv2636hYKkRAJtoLY=; Received: from [74.203.163.58] (port=39893 helo=[10.106.10.38]) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89 (FreeBSD)) (envelope-from ) id 1dkWR4-000AeL-ML; Wed, 23 Aug 2017 09:15:10 -0500 User-Agent: Microsoft-MacOutlook/f.26.0.170815 Date: Wed, 23 Aug 2017 09:14:40 -0500 Subject: Re: ENAMETOOLONG? From: Larry Rosenman To: Martin Simmons CC: Message-ID: <14F658A8-23DB-49DF-9AA3-5BD45BA26BEA@lerctr.org> Thread-Topic: ENAMETOOLONG? References: <20170822141419.v6ely424bjvcnzoj@lrosenman.local> <201708231238.v7NCc4f7008608@higson.cam.lispworks.com> In-Reply-To: <201708231238.v7NCc4f7008608@higson.cam.lispworks.com> Mime-version: 1.0 Content-type: text/plain; charset="UTF-8" Content-transfer-encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 14:15:12 -0000 On 8/23/17, 7:38 AM, "Martin Simmons" wrote: >>>>> On Tue, 22 Aug 2017 09:14:19 -0500, Larry Rosenman said: > > Are there plans to increase the max length for zfs? > > [1239928] ZFS WARNING: Unable to create ZVOL zroot/vm-bhyve/freebsd11/disk0@zfs-auto-snap_hourly-2017-08-22-09h00 (error=63). I think this is hitting the limit on SPECNAMELEN when it tries to create the device node. __Martin Any chance of that limit be increased? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106 From owner-freebsd-fs@freebsd.org Wed Aug 23 18:08:55 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB544DEC6F5 for ; Wed, 23 Aug 2017 18:08:55 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 985427D6EF for ; Wed, 23 Aug 2017 18:08:55 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (mail.distal.com [IPv6:2001:470:e24c:200:0:0:0:ae25]) (authenticated bits=0) by hydra.pix.net (8.16.0.19/8.15.2) with ESMTPSA id v7NI8Y3C098994 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 23 Aug 2017 14:08:42 -0400 (EDT) (envelope-from cross+freebsd@distal.com) Received: from rtp-vpn1-299.cisco.com ([173.38.117.88]) (authenticated bits=0) by mail.distal.com (8.15.2/8.15.2) with ESMTPSA id v7NI8WGx068448 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 23 Aug 2017 14:08:33 -0400 (EDT) (envelope-from cross+freebsd@distal.com) From: Chris Ross Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_82DA429E-BFB5-4362-82E7-AAC6D0939E51"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: zfs import from old TXG Date: Wed, 23 Aug 2017 14:08:17 -0400 In-Reply-To: <1d3771a3-9c3a-1af9-9c48-032a01629630@b1t.name> Cc: freebsd-fs@freebsd.org To: Volodymyr Kostyrko References: <3408A832-BF2E-4525-9EAC-40979BA3555B@distal.com> <20170813000323.GC95431@server.rulingia.com> <5B508AED-4DAC-42E4-8C56-4619C0D1A1C6@distal.com> <035BFBCB-BA43-4568-89E9-8E8DCDFAA8CA@kraus-haus.org> <928F1F16-0777-4D66-BD27-8224DF80363C@distal.com> <06EEEF86-E466-44EA-86F1-866DA32DD92D@kraus-haus.org> <1d3771a3-9c3a-1af9-9c48-032a01629630@b1t.name> X-Mailer: Apple Mail (2.3273) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (mail.distal.com [206.138.151.250]); Wed, 23 Aug 2017 14:08:33 -0400 (EDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 18:08:56 -0000 --Apple-Mail=_82DA429E-BFB5-4362-82E7-AAC6D0939E51 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Aug 13, 2017, at 09:54, Volodymyr Kostyrko wrote: >=20 > Paul Kraus wrote: >=20 >> Using zdb you can examine the TXGs and there is a way to force an = import to a certain TXG (at least under Illumos there is, I assume the = FBSD ZFS code is the same). I do not remember the specific procedure, = but it went by on one of the ZFS mailing lists a few years ago. The goal = at the time was not to deal with a changed pool configuration but a = series of writes that corrupted something (possibly due to a feature = being turned on at a certain point in time). You would lose all data = written after the TXG you force the import to. >>=20 >> Looking at the manpages for zpool and zdb, it looks like doing this = has gotten easier with `zdb -e -F` =E2=80=A6 operate on exported pool = and rollback TXG until the pool is importable =E2=80=A6 but I=E2=80=99m = sure there are limits :-) >=20 > That's -T, it was intentionally left undocumented. >=20 > /usr/src/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c /starting txg Looking back at this, and trying to recover some data without = multi-day clean/restore cycles. I want to import my pool to an earlier = txg. I used zfs history to log the actions, and am trying to use = =E2=80=9Czpool import -N [-F] -T 7428 tank=E2=80=9D. With or without a = -F argument, I get: cannot import 'tank': one or more devices is currently unavailable However, zpool import shows all of its devices ONLINE, and I only = exported it a short while ago. Is there something I=E2=80=99m missing = that I can provide to proceed with effectively importing an earlier = state of this pool? Thanks=E2=80=A6 - Chris --Apple-Mail=_82DA429E-BFB5-4362-82E7-AAC6D0939E51 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZncSWAAoJEPFBDnXvoNg0DXAP/RDDjmCCOR1/5sz4hFBU6js+ B9s7qjj3bM594mR+4YDl6sW9cvHujiLUrxGgLoh7aLlvk8FOm1C6OY+5zT9Ibu2+ aSF4qWUnYi5wakXHXltHt4EJ0OBwsB+r3LjGN2dt7ZAR8FbpoFOY1vU5lCfqkLGp cK+g5chM5hOnA576TJaTELaqHee5/4LkdYgBGdsT+t3wE0jvNDdzT/GHzqsb4EId tw35PLvt2ZpnW5Ei3MJ2mpY5aALIqBv5+uPGGAGbAtCPfmeUZWXL8cCa8B77hU79 2estIhB8N6lWlkMDrkZyxI8Ah5ZhD2SJ66IQexYApuimjDRl59SUSXuOBq2WqDy5 OYQXivjDkMlOMhXRHiAewdBFfajJSown7biSdgXLwt0bfWhKELo3aUv/FinH6ZnF 45jEZH7FHacXbfYAkf7r5rl8I0Se01xUil5snEZ68SMwXG/sCq9ERp4Sogoa0bD5 itA6YzcDHAEvyAr5an+HHUH+jkdhQx8wOwMrIJdXdtS70nBYqNgbFG4tNuKDpO1l Ix9qZmNePIcsoKBb177FCiCwZ8tAiVBnRommuSyuwXw00eUco9X6WqRaSpMCDuKA Fe8jN3kSgQlBsJNOD98JJLVtykHKsv/mq+kXp3iuupzO9KzBKCzREJisrgmgL2/0 tT1JhuvVKi+uyH50BwvT =zuKH -----END PGP SIGNATURE----- --Apple-Mail=_82DA429E-BFB5-4362-82E7-AAC6D0939E51-- From owner-freebsd-fs@freebsd.org Wed Aug 23 18:38:14 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29F2ADECDE1 for ; Wed, 23 Aug 2017 18:38:14 +0000 (UTC) (envelope-from arcade@b1t.name) Received: from limbo.b1t.name (limbo.b1t.name [78.25.32.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6C10F7E209 for ; Wed, 23 Aug 2017 18:38:13 +0000 (UTC) (envelope-from arcade@b1t.name) Received: from [172.29.1.64] (probe.42.lan [172.29.1.64]) by limbo.b1t.name (Postfix) with ESMTPSA id 1A44B15C; Wed, 23 Aug 2017 21:32:02 +0300 (EEST) Subject: Re: zfs import from old TXG To: Chris Ross Cc: freebsd-fs@freebsd.org References: <3408A832-BF2E-4525-9EAC-40979BA3555B@distal.com> <20170813000323.GC95431@server.rulingia.com> <5B508AED-4DAC-42E4-8C56-4619C0D1A1C6@distal.com> <035BFBCB-BA43-4568-89E9-8E8DCDFAA8CA@kraus-haus.org> <928F1F16-0777-4D66-BD27-8224DF80363C@distal.com> <06EEEF86-E466-44EA-86F1-866DA32DD92D@kraus-haus.org> <1d3771a3-9c3a-1af9-9c48-032a01629630@b1t.name> From: Volodymyr Kostyrko Message-ID: Date: Wed, 23 Aug 2017 21:31:59 +0300 User-Agent: Mozilla/5.0 (X11; DragonFly x86_64; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=b1t.name; s=dkim; t=1503513123; bh=txy4pD/Bsiwi+Heehf9SszIgu6de48fDIQEX/wzRP30=; h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=TM+sS7wnXOY47yuJZTZRP+SrEBmn5cnlMO/CM3BuZ3vvPJ6FOTy04ThMIrzx5kWRvHkHvLxsW4sXESvfsuN8TwvOlpbZsZW7Lcwz9feB6U5uI3G3m4gwViuqY2i1bq6bjDgQ4t1PcTxhEYMfGgJxZBxtIALBhgv5gNbPySpoM2U= X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 18:38:14 -0000 Chris Ross wrote: > >> On Aug 13, 2017, at 09:54, Volodymyr Kostyrko wrote:= >> >> Paul Kraus wrote: >> >>> Using zdb you can examine the TXGs and there is a way to force an imp= ort to a certain TXG (at least under Illumos there is, I assume the FBSD = ZFS code is the same). I do not remember the specific procedure, but it w= ent by on one of the ZFS mailing lists a few years ago. The goal at the t= ime was not to deal with a changed pool configuration but a series of wri= tes that corrupted something (possibly due to a feature being turned on a= t a certain point in time). You would lose all data written after the TXG= you force the import to. >>> >>> Looking at the manpages for zpool and zdb, it looks like doing this h= as gotten easier with `zdb -e -F` =E2=80=A6 operate on exported pool and = rollback TXG until the pool is importable =E2=80=A6 but I=E2=80=99m sure = there are limits :-) >> >> That's -T, it was intentionally left undocumented. >> >> /usr/src/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c /starting txg= > > Looking back at this, and trying to recover some data without multi-d= ay clean/restore cycles. I want to import my pool to an earlier txg. I = used zfs history to log the actions, and am trying to use =E2=80=9Czpool = import -N [-F] -T 7428 tank=E2=80=9D. With or without a -F argument, I g= et: > > cannot import 'tank': one or more devices is currently unavailable > > However, zpool import shows all of its devices ONLINE, and I only exp= orted it a short while ago. Is there something I=E2=80=99m missing that = I can provide to proceed with effectively importing an earlier state of t= his pool? Available transactions can be listed with: zdb -ul I can't remember how long it takes to overwrite old transactions but if=20 the pool was imported more then a few hours after the disk was added=20 older transactions would already be overwritten... PS: If not minutes. --=20 Sphinx of black quartz judge my vow. From owner-freebsd-fs@freebsd.org Wed Aug 23 19:27:39 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A473BDED91F for ; Wed, 23 Aug 2017 19:27:39 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3DA157F551 for ; Wed, 23 Aug 2017 19:27:38 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (mail.distal.com [IPv6:2001:470:e24c:200:0:0:0:ae25]) (authenticated bits=0) by hydra.pix.net (8.16.0.19/8.15.2) with ESMTPSA id v7NJRHaw001352 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 23 Aug 2017 15:27:24 -0400 (EDT) (envelope-from cross+freebsd@distal.com) Received: from [IPv6:2001:420:c0c4:1005::2f2] ([IPv6:2001:420:c0c4:1005:0:0:0:2f2]) (authenticated bits=0) by mail.distal.com (8.15.2/8.15.2) with ESMTPSA id v7NJREAU068843 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 23 Aug 2017 15:27:15 -0400 (EDT) (envelope-from cross+freebsd@distal.com) From: Chris Ross Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_92FF6035-E83E-436D-B31A-85FD975AB3FB"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: zfs import from old TXG Date: Wed, 23 Aug 2017 15:27:00 -0400 In-Reply-To: Cc: freebsd-fs@freebsd.org To: Volodymyr Kostyrko References: <3408A832-BF2E-4525-9EAC-40979BA3555B@distal.com> <20170813000323.GC95431@server.rulingia.com> <5B508AED-4DAC-42E4-8C56-4619C0D1A1C6@distal.com> <035BFBCB-BA43-4568-89E9-8E8DCDFAA8CA@kraus-haus.org> <928F1F16-0777-4D66-BD27-8224DF80363C@distal.com> <06EEEF86-E466-44EA-86F1-866DA32DD92D@kraus-haus.org> <1d3771a3-9c3a-1af9-9c48-032a01629630@b1t.name> X-Mailer: Apple Mail (2.3273) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (mail.distal.com [IPv6:2001:470:e24c:200:0:0:0:ae25]); Wed, 23 Aug 2017 15:27:16 -0400 (EDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2017 19:27:39 -0000 --Apple-Mail=_92FF6035-E83E-436D-B31A-85FD975AB3FB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Aug 23, 2017, at 14:31, Volodymyr Kostyrko wrote: >>> That's -T, it was intentionally left undocumented. >>>=20 >>> /usr/src/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c /starting = txg >>=20 >> Looking back at this, and trying to recover some data without = multi-day clean/restore cycles. I want to import my pool to an earlier = txg. I used zfs history to log the actions, and am trying to use = =E2=80=9Czpool import -N [-F] -T 7428 tank=E2=80=9D. With or without a = -F argument, I get: >>=20 >> cannot import 'tank': one or more devices is currently unavailable >>=20 >> However, zpool import shows all of its devices ONLINE, and I only = exported it a short while ago. Is there something I=E2=80=99m missing = that I can provide to proceed with effectively importing an earlier = state of this pool? >=20 > Available transactions can be listed with: >=20 > zdb -ul >=20 > I can't remember how long it takes to overwrite old transactions but = if the pool was imported more then a few hours after the disk was added = older transactions would already be overwritten... >=20 > PS: If not minutes. It was imported, though not mounted, for a few minutes during which = time I broke something I=E2=80=99m trying to recover, and a few minutes = after until I exported it and started evaluating whether it was = recoverable. Using zdb -ul, I see no tag=E2=80=99s around where I was aiming = (7428), but I do see three in the 7300=E2=80=99s. But, trying =E2=80=9Czp= ool import -N -T 7375 tank=E2=80=9D produces the same error. As with = the other two txg=E2=80=99s (7378, 7381) I saw. Any other ideas, since it looks like =E2=80=9Cavailable transaction=E2=80= =9D isn=E2=80=99t my [only] problem? - Chris --Apple-Mail=_92FF6035-E83E-436D-B31A-85FD975AB3FB Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZndcNAAoJEPFBDnXvoNg0mkQP/R2pHRn5AdLp0vSB0lGvzkNm znbSho4H5CENat/ZQz7ug89MELDRC7zyyyKFvdQIs+x1bhZY2p3p/rykkVMWK4Vc /mb2L9H5ZP38u8tZRnpOeLLcbK2I/q6+uy1SoJ5A4rF8s+oS9OMiuWaUuVAlcbB7 lpfFEQQU80WjEq+yom7Et+I1eK0rBqYVeNwmVV8tht6BY/04LrciNuCSwR8y4BfR sGC7PDRwbFBjnmN0hEYIXogsHJJWVY5PgAzyd+3wVAzO085I4gW9lO3wvpGNO1be axRDMuaEzp9wD6if+KRDvDU+U3PEB1DzAvji4Bbr5ykxqaZLqA7v5QAUE2pbMez8 CvlbB0jUzYPVz1WB0XAaFEGFB/TjK2sir6Khi6LDZQ10g3haHfCe68zACG6wq/S2 KXl6kjSKPGDswEXWwjsOczDi12Zd7TvsDREXcDlwpF0VvRcToLV4NlHtZw+ZaND5 Jb+x+jTKVqSAvtvro0tCvkfcrVEXeC7KakbcgXiO7AVoigqr6b/sReE3Rm1szOF2 v4m5dRvBE9LPng2jCNBonLE+mNBIjnYh8UbYB5NeaRQMge+yRf+hun3LnZh+5xLI eVqaJ1WPUBqo9KlySBvaVHFSXxlKFX7AaIqrgGscmbl6kDZGbBlT3MbtVHc87Wpb mvS5JO35c32v0Nm3+q4O =ycmX -----END PGP SIGNATURE----- --Apple-Mail=_92FF6035-E83E-436D-B31A-85FD975AB3FB-- From owner-freebsd-fs@freebsd.org Thu Aug 24 05:40:16 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9325EDD68E7 for ; Thu, 24 Aug 2017 05:40:16 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C1D36B445 for ; Thu, 24 Aug 2017 05:40:16 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: by mail-it0-x22b.google.com with SMTP id j62so1056494ith.1 for ; Wed, 23 Aug 2017 22:40:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=jX3BUVu31pQJHmInOJbOe+3AN6J4jzmNoob5Ngls4a8=; b=cAbaN6/vPEY8zomx0WXFiNjQfjmEfsoURmlQT6Akg5PyGsuoffk80oCRuBc83LU3Wd 7Lke/bV1nIT89gXvSg8b50N5DMg33Gr/rHXBVPcDAnhnLgAK8jO1PvG6wnMJz/kFO+Gs NECo2XGY5qWdcg0baMC5JNfeaMFk9cKYPeQxY2GrN8nX6+3nnRAYbs/842SsJCnEr43Y uj5CTmJSGsyaK7HurVjpt71lUzTAyQf0oApdIZXebteFmzMqS/JJmNShUd3R4+Z5HNUQ UIgBPZaNBD81VPSSgJKU80ZaRyVogrsQh8/amqM2UOLXedLYqlREbZyJcNhURpFzgftB u8Sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=jX3BUVu31pQJHmInOJbOe+3AN6J4jzmNoob5Ngls4a8=; b=KTw88gkf4VAhlyUEj4BOK3qElP0pfHhv0Cqppnd7//OaqrY/5vKaMH4+JzzCxPFix+ lJLLlSRJQuCNmO5nJ55CTMdj4IK3DCauoxkueDHpzMs1CddeWuUnwq29d9fegyGrCzeD 8GDzSYJQ2uc8c5cq7egPGSvZpcKHRGlU8ZKqy5p88ZMTy1i8dPOm6j7/1Y3nX2NHY8sr Ohs3SFQ0HBJDQL7rIILoM2pbtUImfMKt5OqCGVuQTZQHVKOahdJDfwM8AWNAf7wY37Q1 jEmi1Fruxc7/eMIFaYCHW92BXBplyoK6N1cj3bhevXWfQ7TTIo+KPaucXYsu9NxAO3L2 kFTw== X-Gm-Message-State: AHYfb5gaELE24/qdwSCZcz4ez6JYbRI6dBzX6s30QB6DngNRkqDqnS6P 5XCrTIAbgUIZOPJzf9eoCGTK8f+rBA== X-Received: by 10.36.48.16 with SMTP id q16mr4793747itq.112.1503553215353; Wed, 23 Aug 2017 22:40:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.2.200 with HTTP; Wed, 23 Aug 2017 22:40:14 -0700 (PDT) From: Aijaz Baig Date: Thu, 24 Aug 2017 11:10:14 +0530 Message-ID: Subject: Tips on remote debugging for filesystem code To: freebsd-fs@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Aug 2017 05:40:16 -0000 I am trying to understand the internals of the VFS/VNODE interface in the kernel and to that end I was attempting to go through the code flow. Hence I hooked up two FreeBSD VMs, one as the server and the second as the client using named pipes as serial ports. I put a breakpoint on say 'ufs_lookup' and I hit it by something as simple as doing an 'ls' over a directory. Then I try to step through the code and examine how the structures get populated and so on. However when I step through the code on (K)GDB after a few lines of C code, the server VM (the debugged machine) just kind of freezes while the client (on which GDB is run is also waiting on the server) and thereafter I always have to restart the server VM Am I doing something incorrectly? How do you guys normally do it? Keen to hear tips and best practices -- Best Regards, Aijaz Baig From owner-freebsd-fs@freebsd.org Thu Aug 24 08:25:07 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A15A0DD96A7 for ; Thu, 24 Aug 2017 08:25:07 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F0076F29E for ; Thu, 24 Aug 2017 08:25:06 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (124-148-71-183.dyn.iinet.net.au [124.148.71.183]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id v7O8Ou7k014813 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 24 Aug 2017 01:24:59 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: Tips on remote debugging for filesystem code To: Aijaz Baig , freebsd-fs@freebsd.org References: From: Julian Elischer Message-ID: <046d8df4-71a6-65f5-18ad-50589d6d466d@freebsd.org> Date: Thu, 24 Aug 2017 16:24:50 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Aug 2017 08:25:07 -0000 On 24/8/17 1:40 pm, Aijaz Baig wrote: > I am trying to understand the internals of the VFS/VNODE interface in the > kernel and to that end I was attempting to go through the code flow. Hence > I hooked up two FreeBSD VMs, one as the server and the second as the client > using named pipes as serial ports. > > I put a breakpoint on say 'ufs_lookup' and I hit it by something as simple > as doing an 'ls' over a directory. Then I try to step through the code and > examine how the structures get populated and so on. However when I step > through the code on (K)GDB after a few lines of C code, the server VM (the > debugged machine) just kind of freezes while the client (on which GDB is > run is also waiting on the server) and thereafter I always have to restart > the server VM > > Am I doing something incorrectly? How do you guys normally do it? Keen to > hear tips and best practices > I have had more success recently using BHype to make a Virtual FreeBSD machine and connecting to it using the built-in gdb interface. Firstly it is easier than a serial interface and secondly you don't need two machines. From owner-freebsd-fs@freebsd.org Thu Aug 24 09:30:49 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C34FDDDAE8A for ; Thu, 24 Aug 2017 09:30:49 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: from mail-io0-x235.google.com (mail-io0-x235.google.com [IPv6:2607:f8b0:4001:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C08C71449; Thu, 24 Aug 2017 09:30:49 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: by mail-io0-x235.google.com with SMTP id r14so298827iod.1; Thu, 24 Aug 2017 02:30:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Sx4diBTNn8VSqv12FlICaO7skVSoqHTBrQQ6ILw3Hww=; b=M+/gFNFtwPshufKq5wh72yJ6y6p6X3L0WAIfkFV/hy1Khu8cy3kmWX8oAA6U9fIfMQ 4IoQKneLO8nx7h/VeA/+8JWlSjs1GHqd6jnHLdBgoX1QlazJ198p0gyPGNLcpuEcbY/J Z8V3tshNVLOzpUZy+Bt5xAR/AgFyS+Jmh1mT2P08eIp6GM7j4cQCI6NFO2t/NYjNvWH2 x9Jwi4Z162Ih+3W+7fuaNYArM5AUI14XOveyPpYXVUe/Vmg1uCVlb3PytmEtcv4Qk9mU klCMV4m7exEjYlW7g/VK9uEUmdTqXeVkAR1H7BBrwg8Wy5j6Q5ox7fOgabWQq0rEN7hx 9Fxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Sx4diBTNn8VSqv12FlICaO7skVSoqHTBrQQ6ILw3Hww=; b=YiidWzEeatmUBD3oOrvREJqz2TYVuW05RfPW0oGOkWgrzdS0D75E8alvSdjDimSzEi /k4YBhWvUAtTh2g1CIaIBHOjdBckBRrVtS7bue6f3QuM6RmulqP26Haebf4kYobt5WKN wmMvOZGKtSgAVAdP4KEpJcWjTHqBLfKWKAKJrrnIcJ47VSp/pNrv7/L5wrnHn4KyRx59 Gx9YBiEqOPHbBruLy/5QWlicFWTTcV9qDYFwUd63XUDM37LM3ZRrg6irOzkboMcKn+ch mdGKkg+i6xKACkr1YoPEIoX/M7sYHhgDPUvmSSbfETMQc5wSGd0eOaSVRsDNQEaYbO2P xSSw== X-Gm-Message-State: AHYfb5iGDe5Ca14VbSZOyw+a4s8QGdw/LUCamIAaeUpMHyLrwLKx2joh w3teHgX8eNXV2VLRXS2USWeUs5+e6w== X-Received: by 10.107.155.69 with SMTP id d66mr4450322ioe.75.1503567048739; Thu, 24 Aug 2017 02:30:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.2.200 with HTTP; Thu, 24 Aug 2017 02:30:48 -0700 (PDT) In-Reply-To: <046d8df4-71a6-65f5-18ad-50589d6d466d@freebsd.org> References: <046d8df4-71a6-65f5-18ad-50589d6d466d@freebsd.org> From: Aijaz Baig Date: Thu, 24 Aug 2017 15:00:48 +0530 Message-ID: Subject: Re: Tips on remote debugging for filesystem code To: Julian Elischer Cc: freebsd-fs@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Aug 2017 09:30:49 -0000 Can I run BHyve from inside a FreeBSD VM? Will I need to be aware of any pitfalls? Or does it need to actually run on bare metal? On Thu, Aug 24, 2017 at 1:54 PM, Julian Elischer wrote: > On 24/8/17 1:40 pm, Aijaz Baig wrote: > >> I am trying to understand the internals of the VFS/VNODE interface in the >> kernel and to that end I was attempting to go through the code flow. Hence >> I hooked up two FreeBSD VMs, one as the server and the second as the >> client >> using named pipes as serial ports. >> >> I put a breakpoint on say 'ufs_lookup' and I hit it by something as simple >> as doing an 'ls' over a directory. Then I try to step through the code and >> examine how the structures get populated and so on. However when I step >> through the code on (K)GDB after a few lines of C code, the server VM (the >> debugged machine) just kind of freezes while the client (on which GDB is >> run is also waiting on the server) and thereafter I always have to restart >> the server VM >> >> Am I doing something incorrectly? How do you guys normally do it? Keen to >> hear tips and best practices >> >> I have had more success recently using BHype to make a Virtual FreeBSD > machine > > and connecting to it using the built-in gdb interface. > > > Firstly it is easier than a serial interface and secondly you don't need > two machines. > > > > -- Best Regards, Aijaz Baig From owner-freebsd-fs@freebsd.org Thu Aug 24 17:40:27 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C170DE4CB9 for ; Thu, 24 Aug 2017 17:40:27 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46C419B1 for ; Thu, 24 Aug 2017 17:40:26 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (mail.distal.com [IPv6:2001:470:e24c:200:0:0:0:ae25]) (authenticated bits=0) by hydra.pix.net (8.16.0.19/8.15.2) with ESMTPSA id v7OHe15v041088 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 24 Aug 2017 13:40:09 -0400 (EDT) (envelope-from cross+freebsd@distal.com) Received: from [IPv6:2001:420:2710:1330:4019:4cd8:dbbd:e2e3] ([IPv6:2001:420:2710:1330:4019:4cd8:dbbd:e2e3]) (authenticated bits=0) by mail.distal.com (8.15.2/8.15.2) with ESMTPSA id v7OHdxwi075873 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 24 Aug 2017 13:40:00 -0400 (EDT) (envelope-from cross+freebsd@distal.com) From: Chris Ross Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_D0E70563-5EA7-45A5-BEBE-06AB66BC8CD5"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: zfs import from old TXG Date: Thu, 24 Aug 2017 13:39:53 -0400 In-Reply-To: Cc: freebsd-fs@freebsd.org To: Volodymyr Kostyrko References: <3408A832-BF2E-4525-9EAC-40979BA3555B@distal.com> <20170813000323.GC95431@server.rulingia.com> <5B508AED-4DAC-42E4-8C56-4619C0D1A1C6@distal.com> <035BFBCB-BA43-4568-89E9-8E8DCDFAA8CA@kraus-haus.org> <928F1F16-0777-4D66-BD27-8224DF80363C@distal.com> <06EEEF86-E466-44EA-86F1-866DA32DD92D@kraus-haus.org> <1d3771a3-9c3a-1af9-9c48-032a01629630@b1t.name> X-Mailer: Apple Mail (2.3273) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (mail.distal.com [IPv6:2001:470:e24c:200:0:0:0:ae25]); Thu, 24 Aug 2017 13:40:00 -0400 (EDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Aug 2017 17:40:27 -0000 --Apple-Mail=_D0E70563-5EA7-45A5-BEBE-06AB66BC8CD5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Aug 23, 2017, at 15:27, Chris Ross = wrote: >=20 >> On Aug 23, 2017, at 14:31, Volodymyr Kostyrko = wrote: >>> Looking back at this, and trying to recover some data without = multi-day clean/restore cycles. I want to import my pool to an earlier = txg. I used zfs history to log the actions, and am trying to use = =E2=80=9Czpool import -N [-F] -T 7428 tank=E2=80=9D. With or without a = -F argument, I get: >>>=20 >>> cannot import 'tank': one or more devices is currently unavailable >>>=20 >>> However, zpool import shows all of its devices ONLINE, and I only = exported it a short while ago. Is there something I=E2=80=99m missing = that I can provide to proceed with effectively importing an earlier = state of this pool? >>=20 >> Available transactions can be listed with: >>=20 >> zdb -ul >=20 > Using zdb -ul, I see no tag=E2=80=99s around where I was aiming = (7428), but I do see three in the 7300=E2=80=99s. But, trying =E2=80=9Czp= ool import -N -T 7375 tank=E2=80=9D produces the same error. As with = the other two txg=E2=80=99s (7378, 7381) I saw. >=20 > Any other ideas, since it looks like =E2=80=9Cavailable = transaction=E2=80=9D isn=E2=80=99t my [only] problem? Okay. Well, I tried dropping to single-user, rebooting off of a CD, = and all attempts to =E2=80=9Czpool import -T NNNN=E2=80=9D give me that = same error. Without any other ideas, I=E2=80=99m going to punt back to = =E2=80=9Cnuke it from orbit=E2=80=9D and restore the full backup again. = Takes 40+ hours, so was hoping to avoid it with some trickery, but, a = recoverable situation. Thanks. - Chris --Apple-Mail=_D0E70563-5EA7-45A5-BEBE-06AB66BC8CD5 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZnw9sAAoJEPFBDnXvoNg01TAQAJxIrSqhwGHReikkBR4BHurv WqyYoJOoPeqAPZNLk4EqDk84JMfXYP/Ga44WB4QN0ll+g2MxxKNYpc06uUodspZf J4OxmyTZt3AEcKHAD22IhTxvDndv4l5A+fILzXsyA3DU5u+4aFlYoqkbt+0Zguzu CifuWjwJQDec0xB0vCsP1bs6w9tUNxx1nmQU3l2WCqqCeTGfBJQ3nPPbodbT/WSs kksgnqCpAY2D1SnoZUvms+5OR0oI/AU6xrjcd7bL0HAg24Pa+taGcm8EEzRrKIO9 k0/klTUEVy3u0IebOsg8XnM+aQfPoL3V7webfPRf8rDARM9gDOwx6Ybj1Nr6dNo2 KtF18WAxD2l3gIGtFg5HUJyIbpXX+A8PeedtMY2TzkbeaLIZCvz9ub6JlpKhn13I p//O3mtNmLTftqJwk3mV7t/n9NyIAIAC0RqvWOWaEuzgXYMlxvZpmIEy2fpx8tFu NOAuum+cwn8vOuvuZFy6j5vOJyxUOyBVkWiR6uGhs2HIi5/A6weWA4T1gCHieU1C DY2G9bUSlkmAx8bo+S115xUHTw7v5h965UdGr/3Nq6ldVCLehknYfeIlkrBb1Q1b OWbQQQlktjL4dQ0ptFL9DEvbqIVk5QUDbXZJrGqV0274XIt3eePb7r9rGEgw3L/n vQjbx5HAEdDDR+/aNirO =CMJx -----END PGP SIGNATURE----- --Apple-Mail=_D0E70563-5EA7-45A5-BEBE-06AB66BC8CD5-- From owner-freebsd-fs@freebsd.org Thu Aug 24 18:26:19 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 068E7DE58BB for ; Thu, 24 Aug 2017 18:26:19 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [70.36.157.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DE01520EB; Thu, 24 Aug 2017 18:26:18 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (localhost [IPv6:::1]) by chez.mckusick.com (8.15.2/8.15.2) with ESMTP id v7OIU0nj018164; Thu, 24 Aug 2017 11:30:00 -0700 (PDT) (envelope-from mckusick@chez.mckusick.com) Message-Id: <201708241830.v7OIU0nj018164@chez.mckusick.com> From: Kirk McKusick To: Julian Elischer Subject: Re: Tips on remote debugging for filesystem code cc: Aijaz Baig , freebsd-fs@freebsd.org In-reply-to: <046d8df4-71a6-65f5-18ad-50589d6d466d@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <18162.1503599400.1@chez.mckusick.com> Date: Thu, 24 Aug 2017 11:30:00 -0700 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Aug 2017 18:26:19 -0000 > To: Aijaz Baig , freebsd-fs@freebsd.org > Subject: Re: Tips on remote debugging for filesystem code > From: Julian Elischer > Date: Thu, 24 Aug 2017 16:24:50 +0800 > > On 24/8/17 1:40 pm, Aijaz Baig wrote: > >> ... >> How do you guys normally do it? Keen to hear tips and best practices > > I have had more success recently using BHype to make a Virtual FreeBSD > machine and connecting to it using the built-in gdb interface. > > Firstly it is easier than a serial interface and secondly you don't > need two machines. Is there documentation that describes how to do this? I have been using a patched up set of scripts provided by John Baldwin a couple of years ago to use kgdb with a bhyve VM. But if there is an existing way to do it now, I would rather switch to it. Kirk McKusick From owner-freebsd-fs@freebsd.org Thu Aug 24 19:28:06 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F55BDE68FD for ; Thu, 24 Aug 2017 19:28:06 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4344F6333E; Thu, 24 Aug 2017 19:28:05 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 4EBDB20C64; Thu, 24 Aug 2017 15:28:04 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Thu, 24 Aug 2017 15:28:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=WEagL090w/SpzCKrvx2Gktc1HwcOWGV3OJhoPLvTyqI=; b=k3ZsqyEw 5ltsw8391ztkdxYnMnwjPZTqm48JIMT78CI4uy2FKlGDmf+UYWxPlW60LqcgD+3u ArdS33/Ltq52WQXbrfZXcEYdb5TTKzCSA9bADpFPAySAsDP15i90x+qUajJCZPYi 6YI81O0AN8GqWbMy3yKBx6kcm0qKWRfDQKmSxwvkkjLaHjN7t39vOV+4XMOdwHDE P7DdCjnk3EoPcCodFgeh5YXyOVPnc9xLknLP+vdvd9CI5aDF6889CNKkhoWoHT5t OimYgYaPnOhcZvOrgtgrzI3k4NlIh0DGgsPEU1bwqu4RgOTI3I2MWCbNlTwIQyLC APEnRIV9Z0oHtA== X-ME-Sender: X-Sasl-enc: lDnhcnUfNhalAybZ9Pk7G4D6aEI6Iq1vxJR5+YP2e5RC 1503602884 Received: from ebox.rath.org (ebox.rath.org [45.79.69.51]) by mail.messagingengine.com (Postfix) with ESMTPA id 030797E4F0; Thu, 24 Aug 2017 15:28:04 -0400 (EDT) Received: from vostro.rath.org (vostro [192.168.12.4]) by ebox.rath.org (Postfix) with ESMTPS id F1968D5; Thu, 24 Aug 2017 19:28:02 +0000 (UTC) Received: by vostro.rath.org (Postfix, from userid 1000) id 731781027B9; Thu, 24 Aug 2017 21:28:01 +0200 (CEST) From: Nikolaus Rath To: freebsd-fs@freebsd.org, gnn@FreeBSD.org Subject: Re: Status of FUSE in FreeBSD kernel References: <878tiqdjtc.fsf@vostro.rath.org> <20170814060456.GE48929@e-new.0x20.net> Mail-Copies-To: never Mail-Followup-To: freebsd-fs@freebsd.org, gnn@FreeBSD.org Date: Thu, 24 Aug 2017 21:28:01 +0200 In-Reply-To: <20170814060456.GE48929@e-new.0x20.net> (Lars Engels's message of "Mon, 14 Aug 2017 08:04:56 +0200") Message-ID: <87h8ww4vge.fsf@vostro.rath.org> User-Agent: Gnus/5.130014 (Ma Gnus v0.14) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Aug 2017 19:28:06 -0000 Hi George, On Aug 14 2017, Lars Engels wrote: > On Fri, Aug 11, 2017 at 08:46:07PM +0200, Nikolaus Rath wrote: >> Is there some resource that gives an overview of the features supported >> by the FUSE kernel module in FreeBSD? > > George Neville-Neil (CC'ed) wrote FreeBSD's fuse kernel module. Maybe he > can help you. Could you tell me if FreeBSD's fuse module supports any kind of ioctl on files or directories? I am struggling to get the Linux example to work under FreeBSD. Issuing the ioctl always fails with "Inappropriate ioctl for device". Secondly, am I right to assume that in FreeBSD, FUSE filesystems can only be mounted in directories (but not files)? Under Linux, the mountpoint can also be a file. Thanks! -Nikolaus --=20 GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2= =AB From owner-freebsd-fs@freebsd.org Thu Aug 24 20:45:17 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24D57DE7C52 for ; Thu, 24 Aug 2017 20:45:17 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-oi0-f53.google.com (mail-oi0-f53.google.com [209.85.218.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB9E365661; Thu, 24 Aug 2017 20:45:16 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-oi0-f53.google.com with SMTP id j144so5705921oib.1; Thu, 24 Aug 2017 13:45:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:content-transfer-encoding; bh=HzYsMyfG/seO0kdbGVzkaODNPCu5eEvLjOjlFFCSw2o=; b=ABWw0d2bJjLygWO4/evVUA1qmUh9+dTCbcQ/0VWSECp1BEv0Idge7ND0t6LAdELUrE sMKQbApU1JUIHAkGHQioL1wzmYSuvRgZFY9zuKlYdzVXmOCh0pF4h8N/3SwZ1UcjDFyO VD6XcSDxN3jTyODa+EijQFVtm3LU5mbnQpmKlsJ1zuUGRNPSm7If59CAS7I6wbFSPYGe 4y8RVBWPhvJ+HhmOzsSAhccHY/ZT9S8FdJnzyLQE+qdVAi7RqGnkbWxXhB9TmHdQAH/7 bHqXioNXZhNjpzrmriYozBPg3i/NtHDOvUc9gZNvOKVXpILEjEPQs/b2sxKXf4vZc9ys HJjg== X-Gm-Message-State: AHYfb5hMphnCc20HRhhmMyZDMY5HNTpVLQd5t8zA3SHKU9M3LTpJprvY e7xXNw/3uTvvxqkaemM= X-Received: by 10.202.224.69 with SMTP id x66mr10149271oig.291.1503607093503; Thu, 24 Aug 2017 13:38:13 -0700 (PDT) Received: from mail-io0-f172.google.com (mail-io0-f172.google.com. [209.85.223.172]) by smtp.gmail.com with ESMTPSA id 64sm4476022oif.42.2017.08.24.13.38.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 Aug 2017 13:38:13 -0700 (PDT) Received: by mail-io0-f172.google.com with SMTP id k22so2088534iod.2; Thu, 24 Aug 2017 13:38:13 -0700 (PDT) X-Received: by 10.107.175.219 with SMTP id p88mr6101903ioo.219.1503607092780; Thu, 24 Aug 2017 13:38:12 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.2.81.131 with HTTP; Thu, 24 Aug 2017 13:38:12 -0700 (PDT) In-Reply-To: <87h8ww4vge.fsf@vostro.rath.org> References: <878tiqdjtc.fsf@vostro.rath.org> <20170814060456.GE48929@e-new.0x20.net> <87h8ww4vge.fsf@vostro.rath.org> From: Conrad Meyer Date: Thu, 24 Aug 2017 13:38:12 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Status of FUSE in FreeBSD kernel To: freebsd-fs@freebsd.org, "George V. Neville-Neil" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Aug 2017 20:45:17 -0000 FreeBSD only supports version 7.8 of the FUSE protocol. IOCTLs were added in 7.11. Yes, I don't believe FreeBSD supports mountpoints other than directories. This is enforced in vfs_domount_first() with the v_type !=3D VDIR check. Best, Conrad On Thu, Aug 24, 2017 at 12:28 PM, Nikolaus Rath wrote: > Hi George, > > On Aug 14 2017, Lars Engels wrote: >> On Fri, Aug 11, 2017 at 08:46:07PM +0200, Nikolaus Rath wrote: >>> Is there some resource that gives an overview of the features supported >>> by the FUSE kernel module in FreeBSD? >> >> George Neville-Neil (CC'ed) wrote FreeBSD's fuse kernel module. Maybe he >> can help you. > > Could you tell me if FreeBSD's fuse module supports any kind of ioctl on > files or directories? I am struggling to get the Linux example to work > under FreeBSD. Issuing the ioctl always fails with "Inappropriate ioctl > for device". > > Secondly, am I right to assume that in FreeBSD, FUSE filesystems can > only be mounted in directories (but not files)? Under Linux, the > mountpoint can also be a file. > > Thanks! > -Nikolaus > > -- > GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F > > =C2=BBTime flies like an arrow, fruit flies like a Banana.= =C2=AB > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Thu Aug 24 20:45:50 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6572BDE7C9E for ; Thu, 24 Aug 2017 20:45:50 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3AD63656F0 for ; Thu, 24 Aug 2017 20:45:49 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 917FA20BC8 for ; Thu, 24 Aug 2017 16:45:48 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Thu, 24 Aug 2017 16:45:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=NPBUmCQJ98WgYUOg/x bw2Nz5iMQ98sgjAJs5ruYQn/I=; b=TiJgU724tDuq4NNkDvAl8h2VE6X5I0/++5 NUZP44i3yEHN2lm+90FAKNHadY9APzcKBGtdEy7vnt4nGQGB0pNDTzZRW7mDm/8X R8PLwMo4Hw0B8Okg/Jrova4saQfQYdhviuvlgm9wyAcS7sjCQR126WKdLGxCeI01 ulTaZdhZeIH6Mqviw9TZsONxG4DMFThnXBKlznHs611VABZgVv0uISO8hUz3ZLuQ zHHz/eKdi1wjRYENDOIwNBe1WS2BGhiI7kXK63w6l1r+k2m+MunM/emi9OYZ/4aB k1xOByB4EHcNEBtOR9MHx6xxnn+sKmu7TyGpvh/1UWjAI/+LEr6w== X-ME-Sender: X-Sasl-enc: hy2f7RxmTO3HR1ylZx5BNP4XbrnoYGJvdi9QEzy5kbf9 1503607548 Received: from ebox.rath.org (ebox.rath.org [45.79.69.51]) by mail.messagingengine.com (Postfix) with ESMTPA id 4DF877FA7D for ; Thu, 24 Aug 2017 16:45:48 -0400 (EDT) Received: from vostro.rath.org (vostro [192.168.12.4]) by ebox.rath.org (Postfix) with ESMTPS id 5384CD5 for ; Thu, 24 Aug 2017 20:45:47 +0000 (UTC) Received: by vostro.rath.org (Postfix, from userid 1000) id CA50E1029B1; Thu, 24 Aug 2017 22:45:45 +0200 (CEST) From: Nikolaus Rath To: freebsd-fs@freebsd.org Subject: umount() taking minutes for FUSE filesystems Mail-Copies-To: never Mail-Followup-To: freebsd-fs@freebsd.org Date: Thu, 24 Aug 2017 22:45:45 +0200 Message-ID: <87bmn44ruu.fsf@vostro.rath.org> User-Agent: Gnus/5.130014 (Ma Gnus v0.14) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Aug 2017 20:45:50 -0000 Hello, It seems that in some situations, the unmount() system call takes minutes to unmount fuse filesystems. To reproduce, download a Git snapshot of libfuse 3 from https://github.com/libfuse/libfuse and do: $ md build; cd build $ meson .. $ mesonconf -D buildtype=3Ddebug # optional $ ninja Then execute examples/printcap. This will take very long, and all the time is spent in the unmount(mountpoint, MNT_FORCE) call in mount_bsd.c:fuse_kern_unmount(). Does anyone have an idea what might be causing this and how to fix it? My hunch is that it has something to do with libfuse no longer responding to requests at this point. But I'm at my wits end as to why and what to do about it.... Best, -Nikolaus --=20 GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2= =AB From owner-freebsd-fs@freebsd.org Fri Aug 25 01:03:27 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15B87DEC32D for ; Fri, 25 Aug 2017 01:03:27 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: from mail-io0-x234.google.com (mail-io0-x234.google.com [IPv6:2607:f8b0:4001:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF2796D52A; Fri, 25 Aug 2017 01:03:26 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: by mail-io0-x234.google.com with SMTP id s101so3719062ioe.0; Thu, 24 Aug 2017 18:03:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=nRwLU8S71qbkBN+KPmnDdo+HKrqfSMGaDFn5uAcvlMo=; b=WrVHXhxxOgKxjlIZrAWq7n7mGS3h2jQT/g3Zqz57WhfUxCCqcvh1icPJ6U0EzvCgnR lDSsaTXPZyx8xmRn7aK1EVDOVU55wVuKC2Xxbjs0NIh4z37Adt3ywuMMlah7CVdA317p WMIjSlx0yd5Oe+XufTm4HbfBW3uj+usl1UY12TkVGb7ckeleh5/WP2z4I+Q9X1LS8cVe DOdDXK8bEaZW9aGo/hks7LT5PSCsisGHwbNCkxi6xSXrjWi4CHGX5lFn2fXSSDMLzICl 3bFPwlLPgedN9lcrNr8/D90GzHu41pMD46Z3tJ+wTmlhZy+DHXg6il1c3HKBtZXFRx4k qvkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=nRwLU8S71qbkBN+KPmnDdo+HKrqfSMGaDFn5uAcvlMo=; b=B3WaQCJSWpDbxXhXdvY+dfkHo4IpKmXVl8sak5J617CeTL2/6dfHP8jbJ9hEMi4NIC 3FQoPLf88QUh10FqQkzaKT/i6e/RUAzCH8R6d3DjCEgBUNqxw1PIxfPTJQnnIshanU5p amgQcGtE7QDj/x1Uz+khRtSYF8EeBOLw6MfIgd28+buONzKJW/rnZAWiy4soCDQ6wdvN dy6JpoJar5KgMFa8joMgL+aV5qRYybwm1d8oraT4njt6Yx92jvbDZ5FNSh9lo/ou7I9+ +Oe1ss65gwtUrSs24YqYFqVNqKvhmQ5PoLDgjHdV3V2/RWbJmOsl4twSNZZ5FQ47SWch Oiyg== X-Gm-Message-State: AHYfb5ji0mdTzRTzVz+r2Wpk39V0t2n/adLCZHuvY0oPm8AbwEWCrOgE RAhQ8EgOwzUu6VmVi8sbyj0Y75RVog== X-Received: by 10.107.189.7 with SMTP id n7mr6627862iof.36.1503623005641; Thu, 24 Aug 2017 18:03:25 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.2.200 with HTTP; Thu, 24 Aug 2017 18:03:25 -0700 (PDT) In-Reply-To: <201708241830.v7OIU0nj018164@chez.mckusick.com> References: <046d8df4-71a6-65f5-18ad-50589d6d466d@freebsd.org> <201708241830.v7OIU0nj018164@chez.mckusick.com> From: Aijaz Baig Date: Fri, 25 Aug 2017 06:33:25 +0530 Message-ID: Subject: Re: Tips on remote debugging for filesystem code To: Kirk McKusick Cc: Julian Elischer , freebsd-fs@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Aug 2017 01:03:27 -0000 Yes please I would love to hear about this as well. Nonetheless if any one of you Kirk or Julian could help me get started a bit with what how you guys use it (and those scripts that Kirk talks about), it would be really nice. By the way is this bhyve the only way to go for fs code debugging? On Fri, Aug 25, 2017 at 12:00 AM, Kirk McKusick wrote: > > To: Aijaz Baig , freebsd-fs@freebsd.org > > Subject: Re: Tips on remote debugging for filesystem code > > From: Julian Elischer > > Date: Thu, 24 Aug 2017 16:24:50 +0800 > > > > On 24/8/17 1:40 pm, Aijaz Baig wrote: > > > >> ... > >> How do you guys normally do it? Keen to hear tips and best practices > > > > I have had more success recently using BHype to make a Virtual FreeBSD > > machine and connecting to it using the built-in gdb interface. > > > > Firstly it is easier than a serial interface and secondly you don't > > need two machines. > > Is there documentation that describes how to do this? I have been using > a patched up set of scripts provided by John Baldwin a couple of years > ago to use kgdb with a bhyve VM. But if there is an existing way to do > it now, I would rather switch to it. > > Kirk McKusick > -- Best Regards, Aijaz Baig From owner-freebsd-fs@freebsd.org Fri Aug 25 02:48:15 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA167DED8E5 for ; Fri, 25 Aug 2017 02:48:15 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [70.36.157.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CDD306F8CA; Fri, 25 Aug 2017 02:48:15 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (localhost [IPv6:::1]) by chez.mckusick.com (8.15.2/8.15.2) with ESMTP id v7P2pwcl029687; Thu, 24 Aug 2017 19:51:58 -0700 (PDT) (envelope-from mckusick@chez.mckusick.com) Message-Id: <201708250251.v7P2pwcl029687@chez.mckusick.com> From: Kirk McKusick To: Aijaz Baig Subject: Re: Tips on remote debugging for filesystem code cc: jhb@freebsd.org, Julian Elischer , freebsd-fs@freebsd.org In-reply-to: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <29685.1503629518.1@chez.mckusick.com> Content-Transfer-Encoding: quoted-printable Date: Thu, 24 Aug 2017 19:51:58 -0700 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Aug 2017 02:48:16 -0000 > From: Aijaz Baig > Date: Fri, 25 Aug 2017 06:33:25 +0530 > Subject: Re: Tips on remote debugging for filesystem code > To: Kirk McKusick > Cc: Julian Elischer , freebsd-fs@freebsd.org > X-ASK-Info: Message Queued (2017/08/24 18:09:30) > X-ASK-Info: Confirmed by User (2017/08/24 18:40:26) > = > On Fri, Aug 25, 2017 at 12:00 AM, Kirk McKusick = wrote: > = >>> To: Aijaz Baig , freebsd-fs@freebsd.org >>> Subject: Re: Tips on remote debugging for filesystem code >>> From: Julian Elischer >>> Date: Thu, 24 Aug 2017 16:24:50 +0800 >>> >>> On 24/8/17 1:40 pm, Aijaz Baig wrote: >>> >>>> ... >>>> How do you guys normally do it? Keen to hear tips and best practices >>> >>> I have had more success recently using BHype to make a Virtual FreeBSD >>> machine and connecting to it using the built-in gdb interface. >>> >>> Firstly it is easier than a serial interface and secondly you don't >>> need two machines. >> >> Is there documentation that describes how to do this? I have been using >> a patched up set of scripts provided by John Baldwin a couple of years >> ago to use kgdb with a bhyve VM. But if there is an existing way to do >> it now, I would rather switch to it. >> >> Kirk McKusick > = > Yes please I would love to hear about this as well. Nonetheless if > any one of you Kirk or Julian could help me get started a bit with > what how you guys use it (and those scripts that Kirk talks about), > it would be really nice. Below I enclose an email from John Baldwin that got me going on debugging under bhyve. I am copying John as he likely has progressed considerably in the last two years, so can likely provide some more up to date tips. Though as far as I can see, he never added his updates to the bhyve/vmrun.sh script. So I am not sure how Julian has set up his bhyve debugging which is what lead to my initial question. > By the way is this bhyve the only way to go for fs code debugging? > = > Best Regards, > Aijaz Baig You could use multiple machines or run under other hypervisors, but bhyve lets you do everything on one machine and has the benefit of easily integrating with kgdb. Another option is to read the several chapters in my book that describes how the VFS layer works ;-) Kirk McKusick =3D-=3D-=3D From: John Baldwin Date: Tue, 08 Sep 2015 16:53:26 -0700 To: Kirk McKusick Subject: Re: kgdb ported to devel/gdb On Tuesday, September 08, 2015 04:05:06 PM Kirk McKusick wrote: >> From: John Baldwin >> To: freebsd-current@freebsd.org >> Subject: kgdb ported to devel/gdb >> Date: Mon, 31 Aug 2015 14:32:04 -0700 >> = >> Over the past several months I have ported kgdb to the version of >> gdb in ports. I have a pending patch to the gdb port to add fork >> following, but once that is done (and possibly after updating to >> 7.10) I will try to add my existing work as a KGDB option on the >> port. Until such time, you can try the newer kgdb by checking out >> my branch from git. >> = >> Here's my cheat sheet on how to build the newer kgdb. Note that >> if you build a world with my cross-libkvm patches you should get a >> kgdb that can debug i386 cores on amd64 and vice versa. >> = >> All of the targets that the native devel/gdb support have their >> backends ported (so x86, sparc64, powerpc and powerpc64). I have >> not yet ported arm or mips since those don't work for userland yet >> in upstream gdb. I have only compiled non-x86 backends. Testing >> of the new kgdb on sparc64 and powerpc would be appreciated. >> = >> Steps: >> = >> % git clone https://github.com/bsdjhb/gdb.git >> % git checkout freebsd-7.9.1-kgdb >> % fetch http://www.freebsd.org/~jhb/gdb/build >> % pkg install devel/gdb >> = >> # Having gdb installed will mean you get the python bindings in the rig= ht >> # place. >> = >> % pkg install gmake >> = >> # I think this is the only build tool you need? >> = >> % ./build >> % cd obj >> = >> # Replace 'obj' with 'obj.' for all but amd64 >> = >> % gmake >> = >> # ... wait >> = >> You will now have a binary at 'obj/gdb/kgdb'. I just run it from my ob= j >> tree currently when testing. Once it becomes part of the port it will = get >> installed as /usr/local/bin/kgdb791 or some such. > = > This is very helpful, thanks for doing it. > = > I recently dumped my (very ancient) i386 debugging machine and started > debugging in a bhyve virtual machine. Much quicker to boot and test. > But, I lost the ability to remote debug over a serial line. Is there > some way to remote attach to my virtual machine so I can once again > set breakpoints, single step it, etc? Yes, I do this with my VMs (and this is how I tested the i386 <--> amd64 cross debugging). The way I do this is to attach a nmdm device to COM2 (uart1) in my VMs and then run gdb over that. First, I use a patched vmru= n.sh to always create a /dev/nmdm2{A,B} pair of devices attached to COM= 2 on each VM. The 'B' device is used by bhyve, and I use the 'A' device on = the host with 'target remote' in kgdb. My change to vmrun.sh is this: --- /usr/share/examples/bhyve/vmrun.sh 2015-08-15 16:31:57.891092000 -070= 0 +++ bhyve/vmrun.sh 2015-09-08 16:46:05.000000000 -0700 @@ -91,9 +91,13 @@ loader_opt=3D"" bhyverun_opt=3D"-H -A -P" pass_total=3D0 +com2=3D"" -while getopts ac:C:d:e:g:hH:iI:m:p:t: c ; do +while getopts 2:ac:C:d:e:g:hH:iI:m:p:t: c ; do case $c in + 2) + com2=3D${OPTARG} + ;; a) bhyverun_opt=3D"${bhyverun_opt} -a" ;; @@ -269,7 +273,13 @@ i=3D$(($i + 1)) done + if [ -n "${com2}" ]; then + devargs=3D"$devargs -l com2,${com2}" + elif kldstat -qm nmdm; then + devargs=3D"$devargs -l com2,/dev/nmdm${vmname}2B" + fi + ${FBSDRUN} -c ${cpus} -m ${memsize} ${bhyverun_opt} \ -g ${gdbport} \ -s 0:0,hostbridge \ -s 1:0,lpc \ Note that if you have nmdm kldloaded, this version will place a nmdm device on COM2 automatically. Next, I enable gdb on COM2 by adding 'hint.uart.1.flags=3D"0x80"' to /boot/device.hints inside the VM. To test it, I boot the VM and then run 'sysctl debug.kdb.enter=3D1' on the console (or over an ssh session). On the host I run my new kgdb (since it can do i386 cross debugging via remote out of the box). Note that you will need the kernel.debug on the host. I do this by keeping my work tree on my host machine and exporting it via NFS into the guest. I then build the kernel the "old" way using config and make directly so that the kernel.debug is at 'sys/i386/compile//kernel.debug'. You can then run the new kgdb as 'kgdb /path/to/kernel.debug'. In the VM console, run 'gdb' at the db> prompt to switch over to GDB. You can then use 'target remote /dev/nmdm2A' to connect to it. For example, if your VM is named vm0, you would use /dev/nmdmvm02A. -- = John Baldwin From owner-freebsd-fs@freebsd.org Sat Aug 26 07:34:16 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3EAAADF03ED for ; Sat, 26 Aug 2017 07:34:16 +0000 (UTC) (envelope-from holin@iki.fi) Received: from vs23.mail.saunalahti.fi (vs23.mail.saunalahti.fi [193.64.193.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vs23.mail.saunalahti.fi", Issuer "vs23.mail.saunalahti.fi" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 03FEC2CB1 for ; Sat, 26 Aug 2017 07:34:14 +0000 (UTC) (envelope-from holin@iki.fi) Received: from vs23.mail.saunalahti.fi (localhost [127.0.0.1]) by vs23.mail.saunalahti.fi (Postfix) with ESMTP id 6CC0D200A3 for ; Sat, 26 Aug 2017 10:27:14 +0300 (EEST) Received: from gw01.mail.saunalahti.fi (gw01.mail.saunalahti.fi [195.197.172.115]) by vs23.mail.saunalahti.fi (Postfix) with ESMTP id 6215020085 for ; Sat, 26 Aug 2017 10:27:14 +0300 (EEST) Received: from [10.0.0.7] (62-78-248-13.bb.dnainternet.fi [62.78.248.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by gw01.mail.saunalahti.fi (Postfix) with ESMTPSA id 56F67400B9 for ; Sat, 26 Aug 2017 10:27:13 +0300 (EEST) From: Heikki Lindholm Subject: zfs birthtimes cannot be set To: freebsd-fs@freebsd.org Message-ID: <5d975b32-cdec-1a8c-3a6a-c569c8c125a1@iki.fi> Date: Sat, 26 Aug 2017 10:27:12 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Aug 2017 07:34:16 -0000 Hello list, man utimes says: "The access time is set to the value of the first element, and the modification time is set to the value of the second element. For file systems that support file birth (creation) times (such as UFS2), the birth time will be set to the value of the second element if the second element is older than the currently set birth time." On FreeBSD (11.0), I can't set birthtime / crtime on ZFS although files appear to have it. Is it supposed to work? If not, is there a WONTFIX on this one for some particular reason? On Solaris, crtimes can be set via system xattr calls. Please, cc me as I'm not subscribed to the list. Regards, Heikki Lindholm