From owner-freebsd-fs@freebsd.org Sun Dec 11 08:58:05 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D088CC7212F for ; Sun, 11 Dec 2016 08:58:05 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0059.outbound.protection.outlook.com [207.46.100.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C35086C for ; Sun, 11 Dec 2016 08:58:04 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0191.CANPRD01.PROD.OUTLOOK.COM (10.165.218.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.761.9; Sat, 10 Dec 2016 21:23:27 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.0761.020; Sat, 10 Dec 2016 21:23:27 +0000 From: Rick Macklem To: "freebsd-fs@freebsd.org" Subject: patch that fixes server reboot recovery problems for the NFSV4.1 client for review Thread-Topic: patch that fixes server reboot recovery problems for the NFSV4.1 client for review Thread-Index: AQHSUytZi+M4mCebf0iF2f6xxpfr1w== Date: Sat, 10 Dec 2016 21:23:27 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: e379ded0-e4c6-4d04-5698-08d42142c813 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YTXPR01MB0191; x-microsoft-exchange-diagnostics: 1; YTXPR01MB0191; 7:dSa2ptv8/FU1HIK+QU0ih/Q8ZT3D6YW0I5XYL5orXl1F6ur9wxTrTnDVY+kbZ0zCiYf9aOO6AWEIy5ej8XXxzOc47ws0aD3WTINsXrcry5uEyoHG/I+yAh7+5rPL9YBk6AxDc25KNrIpaeFd36Ui3K0WlGPt1O/AJX5qnDw35owLokPrZRpyTqjIlJoc8CzKgkKlgfSihgTZuThHz0G5zRuBqaLAQGRpUhPvkCKaxtaRYJh0Hy9wEMF+lisg63cOmpHJGxgjpq/q5ggbI1Cx4zI8pvUWGXAxiKx1yjqpHPgYku3fBxrH8xRd51Sj7Zo4rTMWB0+VeZwhDXrhqjAsg+fahSM8w6Y9nB37wLZ3h3UOqVpjdOMWuv57J/XHtd65ReqMtHgn5hK6ot+6gqmg3KNZ5qCzWymQ5OderR9otitAvQWwBF/Lydxu8GGxFglfvR02vLPeEfuSIZUHWSCMcg== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123555025)(20161123564025)(20161123562025)(20161123560025)(6072148); SRVR:YTXPR01MB0191; BCL:0; PCL:0; RULEID:; SRVR:YTXPR01MB0191; x-forefront-prvs: 0152EBA40F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(39450400003)(189002)(199003)(8936002)(33656002)(122556002)(5660300001)(101416001)(9686002)(6916009)(50986999)(54356999)(450100001)(3280700002)(97736004)(7696004)(92566002)(110136003)(107886002)(2906002)(8676002)(189998001)(74482002)(305945005)(2351001)(102836003)(106356001)(106116001)(105586002)(74316002)(81156014)(77096006)(3660700001)(38730400001)(86362001)(81166006)(68736007)(5640700002)(6436002)(2501003)(6506006)(2900100001); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0191; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Dec 2016 21:23:27.2099 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0191 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 08:58:05 -0000 There is now a patch at: https://reviews.freebsd.org/D8745 that fixes assorted problems w.r.t. the NFSv4.1 client doing recovery after= a server reboot (reply of NFS4ERR_BAD_SESSION to an RPC). Please test/review/comment on it. It is fairly large, but I do not believe it will affect the NFS client in o= ther ways than the NFSv4.1 recovery. This recovery seldom happens for most servers, but it appears that AmazonEFS is an exception. Thanks, rick From owner-freebsd-fs@freebsd.org Sun Dec 11 15:43:38 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09F56C72122 for ; Sun, 11 Dec 2016 15:43:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D9D801465 for ; Sun, 11 Dec 2016 15:43:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uBBFhbnI022481 for ; Sun, 11 Dec 2016 15:43:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 215119] gmirror stop not possible when labels present Date: Sun, 11 Dec 2016 15:43:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: 000.fbsd@quip.cz X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 15:43:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215119 Miroslav Lachman <000.fbsd@quip.cz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |000.fbsd@quip.cz --- Comment #2 from Miroslav Lachman <000.fbsd@quip.cz> --- I met this many years ago and it was really annoying "feature". I think this problem exist with gjournal too. It would be really nice if this auto-tasting can be disabled by some sysctl. I stopped using labels because of this feature/bug. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sun Dec 11 21:00:07 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7016CC7250B for ; Sun, 11 Dec 2016 21:00:07 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6704D1D47 for ; Sun, 11 Dec 2016 21:00:07 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uBBL01xr056066 for ; Sun, 11 Dec 2016 21:00:07 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201612112100.uBBL01xr056066@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention Date: Sun, 11 Dec 2016 21:00:07 +0000 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 21:00:07 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 140068 | [smbfs] [patch] smbfs does not allow semicolon in Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 203419 | solaris assert: (dn->dn_phys->dn_nlevels == 0 && Open | 211491 | System hangs after "Uptime" on reboot with ZFS 7 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Sun Dec 11 23:06:50 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3898DC72EC7 for ; Sun, 11 Dec 2016 23:06:50 +0000 (UTC) (envelope-from 01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@amazonses.com) Received: from a8-26.smtp-out.amazonses.com (a8-26.smtp-out.amazonses.com [54.240.8.26]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 017687DB for ; Sun, 11 Dec 2016 23:06:49 +0000 (UTC) (envelope-from 01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57; d=tarsnap.com; t=1481497602; h=To:From:Subject:Message-ID:Date:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=bRUZ4MKqNhMAI5qxIkb6ySdOIiDoMZhcRomUdVRfj0A=; b=ktxDoPxs82q0pn7qr5FqWmcPY58IZB45oqWw0QWI6besKUWL0urP6ewUJxCWbwLJ TN9V8MN+5dfZ1oT/429Mq3a5glXFBU1MMdjSwNDG4RUGFG5HFQ/qnD91K/xVCPGyFly XAUd9/mP283FBo/Xz6P/XjRc8Axyw2cDP7rFzOdw= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1481497602; h=To:From:Subject:Message-ID:Date:MIME-Version:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=bRUZ4MKqNhMAI5qxIkb6ySdOIiDoMZhcRomUdVRfj0A=; b=1IgsuVWCxLeND4EcBkJupW5ZW4wr1Jd1kxcwRSlZ6fdx1UrBLTnbmFr5YK44zCc3 kGZ7C59HyfTCrfBzpye+ti8eeTHVfcEwHBPg4KdIhNLoMOcSosAyyMM7BvJFeauJs5G ZjysbkU2rc7ySvUZBZfl8DsPez3Y/TP+jLOhF67s= To: "freebsd-fs@freebsd.org" From: Colin Percival Subject: ESTALE after cwd deleted by same NFS client Message-ID: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> Date: Sun, 11 Dec 2016 23:06:42 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2016.12.11-54.240.8.26 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 23:06:50 -0000 Hi filesystem gurus, If I run the following with /nfs/ being an NFS mount: # mkdir /nfs/foo # touch /nfs/foo/bar # cd /nfs/foo # rm -r /nfs/foo # rm bar Then the final 'rm bar' fails with 'Stale NFS file handle'. This results in 'make buildworld' with an NFS-backed /usr/obj failing during cleandir (which, for some reason, seems to delete the same directories multiple times.) I'm not sure if this is an NFS issue or a VFS issue, nor whether this is intended (but IMHO astonishing) behaviour or a bug. I realize that if the 'rm -r' was performed by a different NFS client, this is the behaviour which would be expected; but it seems to me that when the commands are executed by the same NFS client, it should be possible for the cached file handle for /nfs/foo to be invalidated when /nfs/foo is deleted, in order to return ENOENT instead of ESTALE here. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-fs@freebsd.org Sun Dec 11 23:55:46 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0920FC72356 for ; Sun, 11 Dec 2016 23:55:46 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0070.outbound.protection.outlook.com [207.46.100.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B928F1909 for ; Sun, 11 Dec 2016 23:55:45 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.761.9; Sun, 11 Dec 2016 19:23:51 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.0761.021; Sun, 11 Dec 2016 19:23:51 +0000 From: Rick Macklem To: "freebsd-fs@freebsd.org" CC: "jmader2@gmu.edu" Subject: what should mountd do for NFSv4 only servers? Thread-Topic: what should mountd do for NFSv4 only servers? Thread-Index: AQHSU+NLjj1rWizCIka+uFyjSBl5Xw== Date: Sun, 11 Dec 2016 19:23:51 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: 29d3986a-7201-4c74-3127-08d421fb3d66 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YTXPR01MB0189; x-microsoft-exchange-diagnostics: 1; YTXPR01MB0189; 7:BuNcqTKF0ZP+TAK/ir4pzQHlgjKNtPjCYx6DXyLm5HRDdjCcYDtLUDaTIPnRCfXM2NS4x7mpXYduwO47Ms+tdHL4sLL86vBP3nUge5QdmeQAu10Ec9lGm2c+Bu2DPenLJdEo13qyBlYI3o5JHx3HpUVuCXTDd3A2z8EEabmJ5LNZecwWMHLnuYx7wNWNN/aiB7xrTZv0tYMGHmVBTRV6ZtLIZkVUAom+A+u7irNSmgADXzXiNMIKxlxAmVst2uc2HviEL7QDp98TS4SwR9C929uE38EOcEjZJ7x5DheyeKttmjHZxdc5AYOF40Q86bSjuXjdh7UXMH/wkcd8245uY9LuRFJI3i3DPSiHsX7pLnasvospGWuuDlmOnZLSfu/j1mD8itKPrY4VtjfFodhuF7uPjJTQQT9KsB3DPmMzUI7LQqs4KDV422XR0H1+yoE6iZ27f50nc3PnGYRGkUU7kg== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(265634631926514); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123555025)(20161123564025)(20161123562025)(20161123560025)(6072148); SRVR:YTXPR01MB0189; BCL:0; PCL:0; RULEID:; SRVR:YTXPR01MB0189; x-forefront-prvs: 0153A8321A x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(39450400003)(189002)(199003)(102836003)(5640700002)(6506006)(2906002)(4326007)(3280700002)(86362001)(9686002)(8936002)(92566002)(6436002)(97736004)(77096006)(5660300001)(122556002)(3660700001)(38730400001)(2900100001)(7696004)(110136003)(189998001)(54356999)(50986999)(2501003)(81166006)(81156014)(8676002)(33656002)(105586002)(74316002)(106356001)(106116001)(305945005)(6916009)(74482002)(101416001)(2351001)(8666005)(68736007); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0189; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Dec 2016 19:23:51.4345 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0189 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 23:55:46 -0000 I recall an email discussion with Jason Mader w.r.t. what the behaviour of mountd(8) should be when an NFSv4 only server is being run. (I seem to have misplaced the email, so if I recall incorrectly, I apologiz= e and hope that Jason will correct me.;-) mountd(8) serves the Mount protocol, which is related to NFSv2 and NFSv3, b= ut has nothing to do with NFSv4. - At first glance, that would suggest that mountd(8) shouldn't serve Mount = protocol RPCs at all for a NFSv4 only server. However, this would imply that "showmount -e" would not work. I recall that Jason felt that "showmount -e" is useful, at least when run= locally on the server. As such, how does having mountd(8) service Mount protocol requests only fro= m localhost (127.0.0.1 and ::1) and only handle the Export RPC, when the NFS server is = configured as NFSv4 only via vfs.nfsd.server_min_nfsvers =3D=3D 4? Any comments, concerns or other ideas please, rick From owner-freebsd-fs@freebsd.org Mon Dec 12 05:47:51 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29EDAC7230A for ; Mon, 12 Dec 2016 05:47:51 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-6.mit.edu (dmz-mailsec-scanner-6.mit.edu [18.7.68.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D0E5C126E for ; Mon, 12 Dec 2016 05:47:50 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 12074423-447ff70000005085-91-584e38cd1f85 Received: from mailhub-auth-2.mit.edu ( [18.7.62.36]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id C2.11.20613.DC83E485; Mon, 12 Dec 2016 00:42:37 -0500 (EST) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id uBC5gaOZ007149; Mon, 12 Dec 2016 00:42:37 -0500 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id uBC5gX5G014610 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 12 Dec 2016 00:42:36 -0500 Date: Sun, 11 Dec 2016 23:42:33 -0600 From: Benjamin Kaduk To: Colin Percival Cc: "freebsd-fs@freebsd.org" Subject: Re: ESTALE after cwd deleted by same NFS client Message-ID: <20161212054233.GU8460@kduck.kaduk.org> References: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFIsWRmVeSWpSXmKPExsUixG6nonvWwi/C4OYPFYujHasYLY49/snm wOQx49N8Fo/5z08yBzBFcdmkpOZklqUW6dslcGVc2ZJUsIOzYs6xQ6wNjDvZuxg5OSQETCS+ H1rJ1MXIxSEk0MYkMbPzOwuEs5FR4uDBbYwQzlUmiWcLvjOBtLAIqEpceXcPzGYTUJFo6L7M DGKLCGhKHJy8kg3EZhYwldjcfxusRljAQuJX/3awGl4BY4mPG7vB4kICcRK3z+xig4gLSpyc +YQFoldL4sa/l0A1HEC2tMTyfxwgYU6BeImGr2/BxogKKEs0zHjAPIFRYBaS7llIumchdC9g ZF7FKJuSW6Wbm5iZU5yarFucnJiXl1qka6aXm1mil5pSuokRHKQuyjsYX/Z5H2IU4GBU4uF1 SPWNEGJNLCuuzD3EKMnBpCTKy7gOKMSXlJ9SmZFYnBFfVJqTWnyIUYKDWUmEd6WZX4QQb0pi ZVVqUT5MSpqDRUmc91Kme4SQQHpiSWp2ampBahFMVoaDQ0mCd4I5UKNgUWp6akVaZk4JQpqJ gxNkOA/Q8Aaw4cUFibnFmekQ+VOMilLivNEgzQIgiYzSPLheUBKRyN5f84pRHOgVYd6/pkBV PMAEBNf9CmgwE9Dg5/u8QQaXJCKkpBoYcyJjIsoWdB70Wv+CbfLyHKZZ/383ccQ9X2J378yu OZnTHs4sFbM7V/DwN+OvM8/EXE6+3+h1/I7GxJDaPRsdPDkP7573el1+Wu1RWatl1h++VX2K On1aOv7Hv76vN9wvXjGsL7vyNkaT25vhXN23nc28Vd33D7dEPli8ynb3lB+GtZ0TVHi72pRY ijMSDbWYi4oTARTk4gv9AgAA X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 05:47:51 -0000 On Sun, Dec 11, 2016 at 11:06:42PM +0000, Colin Percival wrote: > Hi filesystem gurus, > > If I run the following with /nfs/ being an NFS mount: > # mkdir /nfs/foo > # touch /nfs/foo/bar > # cd /nfs/foo > # rm -r /nfs/foo > # rm bar > > Then the final 'rm bar' fails with 'Stale NFS file handle'. This results > in 'make buildworld' with an NFS-backed /usr/obj failing during cleandir > (which, for some reason, seems to delete the same directories multiple > times.) > > I'm not sure if this is an NFS issue or a VFS issue, nor whether this is > intended (but IMHO astonishing) behaviour or a bug. > > I realize that if the 'rm -r' was performed by a different NFS client, this > is the behaviour which would be expected; but it seems to me that when the > commands are executed by the same NFS client, it should be possible for the > cached file handle for /nfs/foo to be invalidated when /nfs/foo is deleted, > in order to return ENOENT instead of ESTALE here. Amusingly, this just came up recently: https://www.ietf.org/mail-archive/web/nfsv4/current/msg15115.html (et seq) But I guess you did not specify which version of the NFS protocol you were using... -Ben From owner-freebsd-fs@freebsd.org Mon Dec 12 06:15:16 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DDFBC72AA0 for ; Mon, 12 Dec 2016 06:15:16 +0000 (UTC) (envelope-from 01000158f1abbd72-01e38784-8ba0-4c71-9689-76dac8fece0a-000000@amazonses.com) Received: from a8-26.smtp-out.amazonses.com (a8-26.smtp-out.amazonses.com [54.240.8.26]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 442681E2C for ; Mon, 12 Dec 2016 06:15:15 +0000 (UTC) (envelope-from 01000158f1abbd72-01e38784-8ba0-4c71-9689-76dac8fece0a-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57; d=tarsnap.com; t=1481523314; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=ptG6PjmD6wNNnBAktfUkVTY2G6JDKVxsrFsOGXzzynE=; b=bgdDydnAW76G0UzGSHCcWttsPey9rtRSHvbtXkI9xWTNlW7ghsO6Mso6Do9wCuVw F1wZliRUiGJjXvneAuQ8V0SVqqok9++epNTFm5FFz0mqcwAU5CXphHKpBPOmBvc7T5v Um7WwJ3YBfypSv2nsQ9EYgAua3m9U0POCFmvj0BU= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1481523314; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=ptG6PjmD6wNNnBAktfUkVTY2G6JDKVxsrFsOGXzzynE=; b=IeZrDySAvEaoadDNILi/QHd5w/xxeMppw6ebRvkATWRXsj7vPxLqhB6Iumk+S+wx HDCqV37ieWEIv3kbinJK1K0dxf8mEd4SIJxcLBvBJen5J7VuyzGnZHEAOZnQTPOLA+f V8/erSSWIPc6mAxF/hw7rdVMYUPEW3mxTA3tvYSc= Subject: Re: ESTALE after cwd deleted by same NFS client To: Benjamin Kaduk References: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> <20161212054233.GU8460@kduck.kaduk.org> Cc: "freebsd-fs@freebsd.org" From: Colin Percival Message-ID: <01000158f1abbd72-01e38784-8ba0-4c71-9689-76dac8fece0a-000000@email.amazonses.com> Date: Mon, 12 Dec 2016 06:15:14 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.0 MIME-Version: 1.0 In-Reply-To: <20161212054233.GU8460@kduck.kaduk.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2016.12.12-54.240.8.26 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 06:15:16 -0000 On 12/11/16 21:42, Benjamin Kaduk wrote: > On Sun, Dec 11, 2016 at 11:06:42PM +0000, Colin Percival wrote: >> If I run the following with /nfs/ being an NFS mount: >> # mkdir /nfs/foo >> # touch /nfs/foo/bar >> # cd /nfs/foo >> # rm -r /nfs/foo >> # rm bar >> >> Then the final 'rm bar' fails with 'Stale NFS file handle'. > > Amusingly, this just came up recently: > > https://www.ietf.org/mail-archive/web/nfsv4/current/msg15115.html (et seq) > > But I guess you did not specify which version of the NFS protocol you were > using... I'm using NFSv4.1, but this isn't quite the same... that link refers to having one NFS client remove a file out from underneath a different client, while I'm talking about having an NFS client remove a file from underneath *itself*. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-fs@freebsd.org Mon Dec 12 16:41:20 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EA61C73051 for ; Mon, 12 Dec 2016 16:41:20 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-3.mit.edu (dmz-mailsec-scanner-3.mit.edu [18.9.25.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D41A68DE for ; Mon, 12 Dec 2016 16:41:19 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 1209190e-efbff70000000b7b-6e-584ed1f88394 Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 03.35.02939.8F1DE485; Mon, 12 Dec 2016 11:36:08 -0500 (EST) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id uBCGa7on013484; Mon, 12 Dec 2016 11:36:08 -0500 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id uBCGa47M001725 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 12 Dec 2016 11:36:07 -0500 Date: Mon, 12 Dec 2016 10:36:04 -0600 From: Benjamin Kaduk To: Colin Percival Cc: "freebsd-fs@freebsd.org" Subject: Re: ESTALE after cwd deleted by same NFS client Message-ID: <20161212163603.GV8460@kduck.kaduk.org> References: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> <20161212054233.GU8460@kduck.kaduk.org> <01000158f1abc081-d4eddc58-3b4b-41dd-aa1e-0157d2fad812-000000@email.amazonses.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <01000158f1abc081-d4eddc58-3b4b-41dd-aa1e-0157d2fad812-000000@email.amazonses.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFIsWRmVeSWpSXmKPExsUixCmqrPvjol+EweMWWYujHasYLY49/snm wOQx49N8Fo/5z08yBzBFcdmkpOZklqUW6dslcGV0Pa0sWMxRMe3RR9YGxj1sXYycHBICJhIn 1ywGsrk4hATamCT+7HrLBOFsZJQ4dXwrI0iVkMBVJomDC3JBbBYBVYmHd26DxdkEVCQaui8z g9giApoSByevBJvKLGAqsbn/NhOILSxgIfGrfztYDa+AscSX6RMYIRbcZZR4s20BO0RCUOLk zCcsEM1aEjf+vQRq5gCypSWW/+MACXMKxEu8ePcHbK+ogLJEw4wHzBMYBWYh6Z6FpHsWQvcC RuZVjLIpuVW6uYmZOcWpybrFyYl5ealFusZ6uZkleqkppZsYwUEqybeDcVKD9yFGAQ5GJR5e gU1+EUKsiWXFlbmHGCU5mJREeW0uAIX4kvJTKjMSizPii0pzUosPMUpwMCuJ8IYBY0OINyWx siq1KB8mJc3BoiTOeynTPUJIID2xJDU7NbUgtQgmK8PBoSTBaw7SKFiUmp5akZaZU4KQZuLg BBnOAzRcD2Qxb3FBYm5xZjpE/hSjopQ4bzxIswBIIqM0D64XlEQksvfXvGIUB3pFmNcUpIoH mIDgul8BDWYCGvx8nzfI4JJEhJRUA6PChCC7H8scrVqmJoVqzMn6/faw8gOVvnUrdp2yaPHY Pn9vTpGmVbTdEsHTV+ZNEFRe5NGpouxpFhjj/f/l/7l7XLcsXrzQrXQ+G9N+W99NyTUnkirj 5upz+ditjCySkHc9M0d/e6jSjNKv1wPSN/xJieS4fsXI5X3XWvVtUZedam925DSwH1ViKc5I NNRiLipOBADnGAsB/QIAAA== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 16:41:20 -0000 On Mon, Dec 12, 2016 at 06:15:14AM +0000, Colin Percival wrote: > On 12/11/16 21:42, Benjamin Kaduk wrote: > > On Sun, Dec 11, 2016 at 11:06:42PM +0000, Colin Percival wrote: > >> If I run the following with /nfs/ being an NFS mount: > >> # mkdir /nfs/foo > >> # touch /nfs/foo/bar > >> # cd /nfs/foo > >> # rm -r /nfs/foo > >> # rm bar > >> > >> Then the final 'rm bar' fails with 'Stale NFS file handle'. > > > > Amusingly, this just came up recently: > > > > https://www.ietf.org/mail-archive/web/nfsv4/current/msg15115.html (et seq) > > > > But I guess you did not specify which version of the NFS protocol you were > > using... > > I'm using NFSv4.1, but this isn't quite the same... that link refers to having > one NFS client remove a file out from underneath a different client, while I'm > talking about having an NFS client remove a file from underneath *itself*. I thought the reply mentioned (in passing) the case of a client removing a file out from under itself as well. But, maybe I was reading too fast. -Ben From owner-freebsd-fs@freebsd.org Mon Dec 12 18:19:46 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8E472C748C3 for ; Mon, 12 Dec 2016 18:19:46 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wj0-x233.google.com (mail-wj0-x233.google.com [IPv6:2a00:1450:400c:c01::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1FFCF1F20 for ; Mon, 12 Dec 2016 18:19:46 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wj0-x233.google.com with SMTP id tk12so80082274wjb.3 for ; Mon, 12 Dec 2016 10:19:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-disposition:in-reply-to:user-agent; bh=5rOdGyu1RnO2r4sk5XAB+++1WHkBkgmBvhguKRXjYgQ=; b=LMFLc4+pubfJjDcK0UtRRXjCC4xDrgqHr0Evo6rLFpcFsaD2SzRj47MUjADBHW0glD MdXhOhFT6YoOcjPwinrek2rSfJ/BKPz33g0HNjdmyvcdTS6XcnpH1D/RFrZjYyVKg60e BtSYmEeBuoM23aKeATdeCvNQOFdU1SkLT8cl4Yt8ee6eGpvamSrPcvC08rFpBmMOv/8r GWV8/nmX7rYkH0zEgFxSnJDlGcX3aP0WlTzbiDdEEb2wMIUy6yBGqcFB564g2zME/sB1 XrqgEZOCmJ7neJcdHbIBtFg/uA02hXfQm+n1gcOWcxEHG+1+TKE1bzClBwqGuJyVEGS8 KA/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=5rOdGyu1RnO2r4sk5XAB+++1WHkBkgmBvhguKRXjYgQ=; b=SOAYOXHMGy18B5gNjAd2ObPZ+CtNpbVDZBMmIo4TbcAWRlrRqT3MVoOdyajg8rx9ZP zXwKYCEmh5loNTTMMKt2ZT3c3dsk1yTGruoWg71twgqxVs0U4Ldota82TsE/GfxtP1fZ IiqWTGkbcdxvF6b28ibskTeGo+wjxkc/yWwswVronWOqAHcz86vqZ4k1cHxbUJoXBhdQ 8yTT1QONSWExj/HT1nVmdUa8l0MLMepZQtvLFLiB0TVQq6EtTGd6JhNhKvV5KXMlT9SX WXVt9s81QIzCVtY7U4GpBLbxjw+k0N5dCXHXWtfQfW3qJ+friq/i8v+f31rsBXDxl2vi 64Vw== X-Gm-Message-State: AKaTC02muyywVkW93lMxcU8LY/t0XUK96MAAQ8Wt2VsRmaQZo6bLvJdxRbn96aDNVy1Ztg== X-Received: by 10.194.125.43 with SMTP id mn11mr45611078wjb.14.1481566783851; Mon, 12 Dec 2016 10:19:43 -0800 (PST) Received: from brick (aegh224.neoplus.adsl.tpnet.pl. [79.186.163.224]) by smtp.gmail.com with ESMTPSA id d10sm58320239wja.20.2016.12.12.10.19.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Dec 2016 10:19:43 -0800 (PST) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Mon, 12 Dec 2016 19:19:40 +0100 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: Rick Macklem Cc: "freebsd-fs@freebsd.org" , "jmader2@gmu.edu" Subject: Re: what should mountd do for NFSv4 only servers? Message-ID: <20161212181940.GB8058@brick> Mail-Followup-To: Rick Macklem , "freebsd-fs@freebsd.org" , "jmader2@gmu.edu" References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 18:19:46 -0000 On 1211T1923, Rick Macklem wrote: > I recall an email discussion with Jason Mader w.r.t. what the behaviour of > mountd(8) should be when an NFSv4 only server is being run. > (I seem to have misplaced the email, so if I recall incorrectly, I apologize and hope > that Jason will correct me.;-) > > mountd(8) serves the Mount protocol, which is related to NFSv2 and NFSv3, but has > nothing to do with NFSv4. > - At first glance, that would suggest that mountd(8) shouldn't serve Mount protocol > RPCs at all for a NFSv4 only server. > However, this would imply that "showmount -e" would not work. > I recall that Jason felt that "showmount -e" is useful, at least when run locally on > the server. > > As such, how does having mountd(8) service Mount protocol requests only from localhost > (127.0.0.1 and ::1) and only handle the Export RPC, when the NFS server is configured > as NFSv4 only via vfs.nfsd.server_min_nfsvers == 4? > > Any comments, concerns or other ideas please, rick Another thing the mountd is useful for is the "-hosts" autofs map. Restricting mountd to the localhost would break it. From owner-freebsd-fs@freebsd.org Tue Dec 13 01:20:25 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C85AC74B1B; Tue, 13 Dec 2016 01:20:25 +0000 (UTC) (envelope-from javocado@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD7321D86; Tue, 13 Dec 2016 01:20:24 +0000 (UTC) (envelope-from javocado@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id a197so87424807wmd.0; Mon, 12 Dec 2016 17:20:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=ks5P72YneuHYn5HRAzWlTvlPEI4cfDr1xsM7fBaghwQ=; b=XrZYRe8AsrdpHn6ssu6xKdbY2Cadb0hREIFkxI6Qgi17+qnAT40KDrkaqvwGr4+umi xf9Sk3rc502vu04Xlu4Q2nbV1fpOKxF5dJvqX1ClZ6Wru/qi27gQ6j1FNs7sMFqQd0TW dvT+1S5qjT8zwLV96Vh/oUQWS2kxrwhxGShYD93nM6b/D9/iouu5sVXEHxKxRZfJeX90 /yo+Vd0zsrPgHepyTq+mib7X5IT7hTNq658J6doCVwzolHuLkA0nrMNUujEB13vG5kzC NI63Eqnucba1QIewAtuhsLFafIdPkXAWNhy6nPim9BH0nJ1jSXc3388/gBzbYwDZD2Nc W0aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=ks5P72YneuHYn5HRAzWlTvlPEI4cfDr1xsM7fBaghwQ=; b=LUHdUH/NYY5TOVSd/moljcaSHYwHURY0rw1BS8FHKPsIpAJtPb7XUMC0ZCNrl2suRV csJQvE2I+O3LYVTt+TV1oG/3IznpM48CY2XRDl4xLoFPU7NzysF/nnul86ReNWWhCCFz O2uz/9e7g/syWrwm9Xz6hy+r0hoJ76HjlhfAOg554xM0nGU6bjnWFj7uApj/SaYpcoIM waky9PqJy8ZRPSlj9GCz+gVksw0ZOk696tvszBa5JRaeQqkNElBXTP/iBCDOslkrV7/7 AgPKeIAZfGrEPGfowax89X0qImI+5g5srmpc6Hj34pLQGKk8sADOdsGNESndiQRLg7QI vF2Q== X-Gm-Message-State: AKaTC00qxDcq8AizXXUhxeda18PYykcQy+GWHBierxDmTGQfuJouQq/E7m/UW/IfNXmUaucpwO0fLo+arFuLig== X-Received: by 10.46.77.17 with SMTP id a17mr40522055ljb.34.1481592021762; Mon, 12 Dec 2016 17:20:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.114.25.165 with HTTP; Mon, 12 Dec 2016 17:20:21 -0800 (PST) From: javocado Date: Mon, 12 Dec 2016 17:20:21 -0800 Message-ID: Subject: Re-sparse a file-backed IO device + zfs To: freebsd-virtualization@freebsd.org, freebsd-questions@freebsd.org, FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 01:20:25 -0000 Hi, I'm setting up a bhyve wherein: host # truncate -s 1T vol.file host # du -ah vol.file 200K vol.file host # /usr/sbin/bhyve ... -s 4,ahci-hd,vol.file ... Then inside the bhyve I create a zpool (ada0 = vol.file): bhyve # zpool create -O devices=off -O atime=off -O compression=on -m /mnt/data1 data1 ada0 And I put a bunch of stuff in the zpool ... and the vol.file grows in size: host # du -ah vol.file 100G vol.file Then I remove the files from the zpool and the zpool usage returns to 0 but of course the vol.file size does not shrink, the data is still there (but not referenced?) Normally I'd just write zeros to a file inside the zpool until the pool fills up, then maybe cp --sparse vol.file for good measure, but with compression on in the zpool the zeroing doesn't really fill up space or seem to overwrite anything. In my testing the zero file grew larger than 100G with no change to vol.file I did not let it run forever, however. Any other ideas how to scrub off or clear out deleted data from a zpool and/or this kind of file-backed device? From owner-freebsd-fs@freebsd.org Tue Dec 13 01:25:22 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1519C74FB9 for ; Tue, 13 Dec 2016 01:25:22 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qt0-x22d.google.com (mail-qt0-x22d.google.com [IPv6:2607:f8b0:400d:c0d::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 76C01361 for ; Tue, 13 Dec 2016 01:25:22 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-qt0-x22d.google.com with SMTP id w33so93652429qtc.3 for ; Mon, 12 Dec 2016 17:25:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=Z3BWObFzyIeKTSrXcM227IZWryAiGZu/YM2FEOe0isY=; b=uatTZEL8zukKy4INkDUSWLfERUUjMdVhh7SMV2hO/EjIXEZAu+I+TLZyAh0bzqBxE1 R7hwN7ObWr7ECBFqR2lx+7bvB0K1x4+86h/v0NM9w6Os/1m/Y032EhxG3epe7AtFjQ6N JM/MSvfd5bcY/PIg5WhfsevXUm8CNyAkk43IfQzmDMHu56nUXXHi+L6pLl8ScJElwqp5 oBvC+2Vtd3U1i57WDqNcD/7ZZDmcxfP0wJhOEXjhQaLe+7YxvtxKAbu22yBrUrYMtTun 4N+JTrpAQZYKrMImyuDtr0hS7ugHVB12AluZvO146ebMkC1VDhOr0mFSmOYkYXnT0f+8 4Hew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=Z3BWObFzyIeKTSrXcM227IZWryAiGZu/YM2FEOe0isY=; b=V8iJb9hhb+PejQmlNZ+j37QBOwBPQ/mCpTfD1pXv/IckPSQI/qb9V/1/QpglMaZUxV C/Koe1o1vmMlASBWWJit9zeIh6LRkCsC8+9LwPJAzy4VZm0xg4kxHCU2hHv8IYZoa6O6 UhhL7bL3C4dOFWvVpaYJb1FRYLJOR8nzCVp4GaMmFgIQ8DD4GEwYdo9UoiAwMDHhRyqy QglrPLI7aHQPSx8yVfOXoXfhVL1DPjulnhMLEyYTkwemaiFj/aRVJ8ZvM+n75SmU568U w13KZYhjSeW7+Ur11d+EeYISvb4TJgy24gErXfZuX4qxoo1ylKKKsrAtEfK9TB9fjjQ7 5fPA== X-Gm-Message-State: AKaTC03SfbiZ6lrp2t22wQjrJNxBq6WFO2gTVrercnMDqQ74kFf/8TvV4en6uZMSZWoWMHBn X-Received: by 10.237.32.205 with SMTP id 71mr93387527qtb.237.1481592321577; Mon, 12 Dec 2016 17:25:21 -0800 (PST) Received: from mutt-hardenedbsd (pool-100-16-218-231.bltmmd.fios.verizon.net. [100.16.218.231]) by smtp.gmail.com with ESMTPSA id v18sm27867220qkb.40.2016.12.12.17.25.20 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Dec 2016 17:25:20 -0800 (PST) Date: Mon, 12 Dec 2016 20:25:18 -0500 From: Shawn Webb To: javocado Cc: freebsd-virtualization@freebsd.org, freebsd-questions@freebsd.org, FreeBSD Filesystems Subject: Re: Re-sparse a file-backed IO device + zfs Message-ID: <20161213012518.GA77233@mutt-hardenedbsd> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="J/dobhs11T7y2rNN" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD mutt-hardenedbsd 12.0-CURRENT-HBSD FreeBSD 12.0-CURRENT-HBSD X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0x6A84658F52456EEE User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 01:25:22 -0000 --J/dobhs11T7y2rNN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Dec 12, 2016 at 05:20:21PM -0800, javocado wrote: > Hi, >=20 > I'm setting up a bhyve wherein: >=20 > host # truncate -s 1T vol.file > host # du -ah vol.file > 200K vol.file >=20 > host # /usr/sbin/bhyve ... -s 4,ahci-hd,vol.file ... >=20 > Then inside the bhyve I create a zpool (ada0 =3D vol.file): >=20 > bhyve # zpool create -O devices=3Doff -O atime=3Doff -O compression=3Don= -m > /mnt/data1 data1 ada0 >=20 > And I put a bunch of stuff in the zpool ... and the vol.file grows in siz= e: >=20 > host # du -ah vol.file > 100G vol.file >=20 > Then I remove the files from the zpool and the zpool usage returns to 0 b= ut > of course the vol.file size does not shrink, the data is still there (but > not referenced?) >=20 > Normally I'd just write zeros to a file inside the zpool until the pool > fills up, then maybe cp --sparse vol.file for good measure, but with > compression on in the zpool the zeroing doesn't really fill up space or > seem to overwrite anything. In my testing the zero file grew larger than > 100G with no change to vol.file I did not let it run forever, however. >=20 > Any other ideas how to scrub off or clear out deleted data from a zpool > and/or this kind of file-backed device? Instead of dd'ing /dev/zero, try /dev/random. All zeros compress extremely well, [pseudo-]random data does (or, ideally, should) not. --=20 Shawn Webb Cofounder and Security Engineer HardenedBSD GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE --J/dobhs11T7y2rNN Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYT038AAoJEGqEZY9SRW7u1g0P/impDd7fVDSFcO5i/3wlf8M6 8tUs1sg53pJYSNDtEbIfiSODiJDwaXl4+Kdp7o4WbFP68HCzhHPr/qKB4GTyhSPD CgQRa9Jj90E+B7+/zSTba/b5axa30mfSeoVc0Ma1+id/Yq7J+rVEa951eXGCaE/5 bmkrPnWHilYfdRJLY2npVoVwjC44vxn3f1GlD06rrwTV+JCMaw6f7k6hkuAh9trg UyoCJftEH1hzcpyAWYVS4Wn+t/6bXFICKv+tpwDwm+epVkf6tIKvNNFEYxE+TFik PhTk5+i7Lz/12bR6Vh/hKiYKKxHa2rtYSA49TDBMeInOa5yCGQ6f98/Nz/MKc8d9 LUXUSovnVLZkMVObUK31dPF9TuZESK3SGRUTN6CzDArRIcl63xqRWevuv6Hk46Je vurBhvtN89ROzd/BZ10rISuGv+TlyY5MvykggZ7v1obB7nkukFnGQ3PKwmL2DHP5 h1AMc+NFQOU6ym/qcMVDqX/BBuzFBvkgLzZyOjh5wPNsq1ScuBBYO7yFBBwzcQRl rDLviLKBsUNsjHtOF38evqcbGYl19th92o6q4pWiE6hriO/WnaI6yI5WtfPHV7ta oefL87gBeb98YegBF2cCMmE+e0kulJta8qFZQC5bLcIQoBF8Nk9fBfMHUYiGlBxZ /iLIWlsGHOIQt86ev4Nt =wvAw -----END PGP SIGNATURE----- --J/dobhs11T7y2rNN-- From owner-freebsd-fs@freebsd.org Tue Dec 13 07:10:01 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E48EC75786; Tue, 13 Dec 2016 07:10:01 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from EXCH2-1.slu.se (pop.slu.se [77.235.224.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 98966891; Tue, 13 Dec 2016 07:10:00 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by EXCH2-1.slu.se (77.235.224.121) with Microsoft SMTP Server (TLS) id 15.0.1236.3; Tue, 13 Dec 2016 07:54:47 +0100 Received: from exch2-4.slu.se ([fe80::d80d:2775:84ad:c144]) by exch2-4.slu.se ([fe80::d80d:2775:84ad:c144%22]) with mapi id 15.00.1236.000; Tue, 13 Dec 2016 07:54:47 +0100 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: javocado CC: "freebsd-questions@freebsd.org" , "freebsd-virtualization@freebsd.org" , FreeBSD Filesystems Subject: Re: Re-sparse a file-backed IO device + zfs Thread-Topic: Re-sparse a file-backed IO device + zfs Thread-Index: AQHSVQ3KGOl7buWA6UuHYaZ3yCwuRA== Date: Tue, 13 Dec 2016 06:54:46 +0000 Message-ID: Accept-Language: sv-SE, en-US Content-Language: sv-SE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 07:10:01 -0000 DQpEZW4gMTMgZGVjLiAyMDE2IDAyOjIwIHNrcmV2IGphdm9jYWRvIDxqYXZvY2Fkb0BnbWFpbC5j b20+Og0KPg0KPiBIaSwNCj4NCj4gSSdtIHNldHRpbmcgdXAgYSBiaHl2ZSB3aGVyZWluOg0KPg0K PiBob3N0ICMgIHRydW5jYXRlIC1zIDFUIHZvbC5maWxlDQo+IGhvc3QgIyAgZHUgLWFoIHZvbC5m aWxlDQo+IDIwMEsgICAgdm9sLmZpbGUNCj4NCj4gaG9zdCAjICAvdXNyL3NiaW4vYmh5dmUgLi4u IC1zIDQsYWhjaS1oZCx2b2wuZmlsZSAuLi4NCj4NCj4gVGhlbiBpbnNpZGUgdGhlIGJoeXZlIEkg Y3JlYXRlIGEgenBvb2wgKGFkYTAgPSB2b2wuZmlsZSk6DQo+DQo+IGJoeXZlICMgIHpwb29sIGNy ZWF0ZSAtTyBkZXZpY2VzPW9mZiAtTyBhdGltZT1vZmYgLU8gY29tcHJlc3Npb249b24gLW0NCj4g L21udC9kYXRhMSBkYXRhMSBhZGEwDQo+DQo+IEFuZCBJIHB1dCBhIGJ1bmNoIG9mIHN0dWZmIGlu IHRoZSB6cG9vbCAuLi4gYW5kIHRoZSB2b2wuZmlsZSBncm93cyBpbiBzaXplOg0KPg0KPiBob3N0 ICMgIGR1IC1haCB2b2wuZmlsZQ0KPiAxMDBHICAgIHZvbC5maWxlDQo+DQo+IFRoZW4gSSByZW1v dmUgdGhlIGZpbGVzIGZyb20gdGhlIHpwb29sIGFuZCB0aGUgenBvb2wgdXNhZ2UgcmV0dXJucyB0 byAwIGJ1dA0KPiBvZiBjb3Vyc2UgdGhlIHZvbC5maWxlIHNpemUgZG9lcyBub3Qgc2hyaW5rLCB0 aGUgZGF0YSBpcyBzdGlsbCB0aGVyZSAoYnV0DQo+IG5vdCByZWZlcmVuY2VkPykNCj4NCj4gTm9y bWFsbHkgSSdkIGp1c3Qgd3JpdGUgemVyb3MgdG8gYSBmaWxlIGluc2lkZSB0aGUgenBvb2wgdW50 aWwgdGhlIHBvb2wNCj4gZmlsbHMgdXAsIHRoZW4gbWF5YmUgY3AgLS1zcGFyc2Ugdm9sLmZpbGUg Zm9yIGdvb2QgbWVhc3VyZSwgYnV0IHdpdGgNCj4gY29tcHJlc3Npb24gb24gaW4gdGhlIHpwb29s IHRoZSB6ZXJvaW5nIGRvZXNuJ3QgcmVhbGx5IGZpbGwgdXAgc3BhY2Ugb3INCj4gc2VlbSB0byBv dmVyd3JpdGUgYW55dGhpbmcuIEluIG15IHRlc3RpbmcgdGhlIHplcm8gZmlsZSBncmV3IGxhcmdl ciB0aGFuDQo+IDEwMEcgd2l0aCBubyBjaGFuZ2UgdG8gdm9sLmZpbGUgIEkgZGlkIG5vdCBsZXQg aXQgcnVuIGZvcmV2ZXIsIGhvd2V2ZXIuDQo+DQo+IEFueSBvdGhlciBpZGVhcyBob3cgdG8gc2Ny dWIgb2ZmIG9yIGNsZWFyIG91dCBkZWxldGVkIGRhdGEgZnJvbSBhIHpwb29sDQo+IGFuZC9vciB0 aGlzIGtpbmQgb2YgZmlsZS1iYWNrZWQgZGV2aWNlPw0KDQpXb3VsZG4ndCB5b3UgaGF2ZSBuZWVk ZWQgVFJJTSBvbiB0aGUgZGlzayhzKSBpbnNpZGUgdGhlIFZNIGFuZCBpdCB3b3VsZCBoYXZlIHRh a2VuIGNhcmUgb2YgYWxsIHRoaXMgZm9yIHlvdT8NCg0KL0sNCg0KPiBfX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KPiBmcmVlYnNkLWZzQGZyZWVic2Qub3Jn IG1haWxpbmcgbGlzdA0KPiBodHRwczovL2xpc3RzLmZyZWVic2Qub3JnL21haWxtYW4vbGlzdGlu Zm8vZnJlZWJzZC1mcw0KPiBUbyB1bnN1YnNjcmliZSwgc2VuZCBhbnkgbWFpbCB0byAiZnJlZWJz ZC1mcy11bnN1YnNjcmliZUBmcmVlYnNkLm9yZyINCg== From owner-freebsd-fs@freebsd.org Tue Dec 13 09:59:39 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3DE94C70594 for ; Tue, 13 Dec 2016 09:59:39 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from smtp.rlwinm.de (smtp.rlwinm.de [IPv6:2a01:4f8:201:31ef::e]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0BE301FE9 for ; Tue, 13 Dec 2016 09:59:39 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from crest.local (unknown [87.253.189.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.rlwinm.de (Postfix) with ESMTPSA id E31245AF0 for ; Tue, 13 Dec 2016 10:59:36 +0100 (CET) Subject: Re: Re-sparse a file-backed IO device + zfs To: freebsd-fs@freebsd.org References: From: Jan Bramkamp Message-ID: <43b81c7e-8a7b-73f4-0a13-5c91c92663b6@rlwinm.de> Date: Tue, 13 Dec 2016 10:59:36 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 09:59:39 -0000 On 13/12/2016 07:54, Karli Sjöberg wrote: > > Den 13 dec. 2016 02:20 skrev javocado : >> >> Hi, >> >> I'm setting up a bhyve wherein: >> >> host # truncate -s 1T vol.file >> host # du -ah vol.file >> 200K vol.file >> >> host # /usr/sbin/bhyve ... -s 4,ahci-hd,vol.file ... >> >> Then inside the bhyve I create a zpool (ada0 = vol.file): >> >> bhyve # zpool create -O devices=off -O atime=off -O compression=on -m >> /mnt/data1 data1 ada0 >> >> And I put a bunch of stuff in the zpool ... and the vol.file grows in size: >> >> host # du -ah vol.file >> 100G vol.file >> >> Then I remove the files from the zpool and the zpool usage returns to 0 but >> of course the vol.file size does not shrink, the data is still there (but >> not referenced?) >> >> Normally I'd just write zeros to a file inside the zpool until the pool >> fills up, then maybe cp --sparse vol.file for good measure, but with >> compression on in the zpool the zeroing doesn't really fill up space or >> seem to overwrite anything. In my testing the zero file grew larger than >> 100G with no change to vol.file I did not let it run forever, however. >> >> Any other ideas how to scrub off or clear out deleted data from a zpool >> and/or this kind of file-backed device? > > Wouldn't you have needed TRIM on the disk(s) inside the VM and it would have taken care of all this for you? The ahci-hd driver supports TRIM on ZFS volumes. Why do you use sparse file instead of a ZFS volume? From owner-freebsd-fs@freebsd.org Tue Dec 13 13:36:12 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2D3B6C75B8F for ; Tue, 13 Dec 2016 13:36:12 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0067.outbound.protection.outlook.com [157.56.110.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9AEB1D5D for ; Tue, 13 Dec 2016 13:36:11 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQBPR01MB0180.CANPRD01.PROD.OUTLOOK.COM (10.169.141.138) by YQBPR01MB0177.CANPRD01.PROD.OUTLOOK.COM (10.169.141.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.771.8; Tue, 13 Dec 2016 13:04:06 +0000 Received: from YQBPR01MB0180.CANPRD01.PROD.OUTLOOK.COM ([10.169.141.138]) by YQBPR01MB0180.CANPRD01.PROD.OUTLOOK.COM ([10.169.141.138]) with mapi id 15.01.0771.014; Tue, 13 Dec 2016 13:04:06 +0000 From: Rick Macklem To: Benjamin Kaduk , Colin Percival CC: "freebsd-fs@freebsd.org" Subject: Re: ESTALE after cwd deleted by same NFS client Thread-Topic: ESTALE after cwd deleted by same NFS client Thread-Index: AQHSVANFVKFixagVOUGZ8mLDXQwbl6EDzQuAgAC4aM2AAVLGrA== Date: Tue, 13 Dec 2016 13:04:06 +0000 Message-ID: References: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> <20161212054233.GU8460@kduck.kaduk.org> <01000158f1abc081-d4eddc58-3b4b-41dd-aa1e-0157d2fad812-000000@email.amazonses.com>, <20161212163603.GV8460@kduck.kaduk.org> In-Reply-To: <20161212163603.GV8460@kduck.kaduk.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: d24c79e7-655e-4b2f-ecf8-08d42358854a x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YQBPR01MB0177; x-microsoft-exchange-diagnostics: 1; YQBPR01MB0177; 7:N9vRmQ6EI07LkTzgEi5G+j9BIdiSexSeRVTrRfCD3n1tmXVQpdERlc9CvKntqj72Y6WQYcLuwA+u0ZC7j7tLTVcexcdOacWj7gzglFXmAhsz1QMIoYdOzJG3FyWfWENTF1D5xL8VdBR1OnqjeakM3olxowuKl3U4r1D+GUlFXWyKAdNqkz0sMYdLqPz2WsdJWlvtqScf1nZnzAkieoiMjvd7jYg9WM/REgO5TRM/fDSQTQ/sg1EXHvACCxVok46w8ycalGdGqpOwY4pq6IoGQ4KIN+ujNc0XzZPR2aaIBmEywaRe+0XCqmv6ETCw4ai+yT28MghoUNCLfqkHMasw0AUW5RC3dnYWzSTHcWwlFbS6fRokytZKaW1CC7gEspV/7a4RY6Sjw7LNsHdGm9jGdNkbqWvvMDwGnS44dKpNfzGptbA62jBPxQB7PgVM12HZZ26BhxDqBn+gXDBub6WBcg== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123555025)(20161123560025)(20161123564025)(20161123562025)(6072148); SRVR:YQBPR01MB0177; BCL:0; PCL:0; RULEID:; SRVR:YQBPR01MB0177; x-forefront-prvs: 01559F388D x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(39450400003)(189002)(24454002)(199003)(51444003)(9686002)(3280700002)(93886004)(74482002)(3660700001)(305945005)(4326007)(2906002)(2900100001)(92566002)(2171001)(102836003)(81166006)(81156014)(8676002)(8936002)(68736007)(5660300001)(74316002)(7696004)(86362001)(122556002)(2950100002)(106356001)(105586002)(106116001)(38730400001)(77096006)(229853002)(189998001)(54356999)(5001770100001)(97736004)(50986999)(76176999)(33656002)(101416001)(6506006)(6436002); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR01MB0177; H:YQBPR01MB0180.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2016 13:04:06.0421 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR01MB0177 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 13:36:12 -0000 Benjamin Kaduk wrote: >On Mon, Dec 12, 2016 at 06:15:14AM +0000, Colin Percival wrote: >> On 12/11/16 21:42, Benjamin Kaduk wrote: >> > On Sun, Dec 11, 2016 at 11:06:42PM +0000, Colin Percival wrote: >> >> If I run the following with /nfs/ being an NFS mount: >> >> # mkdir /nfs/foo >> >> # touch /nfs/foo/bar >> >> # cd /nfs/foo >> >> # rm -r /nfs/foo >> >> # rm bar If this is happening on a single client, something is broken. - For this step, the name cache should have a miss and the Lookup should fa= il with ENOENT. I think the most likely failures are... --> Either the name cache in the client is broken and has a hit OR --> The server replies NFS_OK for the lookup, even though it should have = been removed. When I get home on the weekend, I'll try this against a FreeBSD and Linux s= erver, to see if it fails. If it does, then I'll try to figure out why. (Version of NFS shouldn't matter for this case. They recently added the opt= ional case of a server that can retain files after an NFSv4 Remove until the NFSv4 Cl= ose, making it more POSIX like. I think that's NFSv4.2, but it might be in NFSV4.1.) Did you try: sysctl debug.disable_vnfullpath=3D1 (or something close to disable_vnfullpa= th) to see if it helped? rick >> >> >> >> Then the final 'rm bar' fails with 'Stale NFS file handle'. >> > >> > Amusingly, this just came up recently: >> > >> > https://www.ietf.org/mail-archive/web/nfsv4/current/msg15115.html (et = seq) >> > >> > But I guess you did not specify which version of the NFS protocol you = were >> > using... >> >> I'm using NFSv4.1, but this isn't quite the same... that link refers to = having >> one NFS client remove a file out from underneath a different client, whi= le I'm >> talking about having an NFS client remove a file from underneath *itself= *. > >I thought the reply mentioned (in passing) the case of a client removing a >file out from under itself as well. But, maybe I was reading too fast. > >-Ben From owner-freebsd-fs@freebsd.org Tue Dec 13 14:09:55 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 816E2C74EE3 for ; Tue, 13 Dec 2016 14:09:55 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from mail-oi0-x22b.google.com (mail-oi0-x22b.google.com [IPv6:2607:f8b0:4003:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4591F18D2 for ; Tue, 13 Dec 2016 14:09:55 +0000 (UTC) (envelope-from dfr@rabson.org) Received: by mail-oi0-x22b.google.com with SMTP id b126so123644263oia.2 for ; Tue, 13 Dec 2016 06:09:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rabson-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=+NdinlcwIipK2AxQ+52YkcDL3y172tlBEWhb8NtIA2M=; b=WM1HJHklL63PxUWvAm2MgB6DogOLcerulK4hphHD+cBQQAzlZPLdfjyp9U2vdaUUpc p/YeGlKosT6YT/EEZ/RKhYsSs+DrRgs+4Z3vu5B6bogNwuG/I9e8CromCPDLNYWotovj 2+uJpVipFJOmJCTmEtvyHGF/WqwTlw+zVRJpGX09/CQaR6/0PbGf3bDX+I91N35qYKxb FKjLLUvvWkoLGTJoFai0vCiNZGDAAYK2DPOD97cx6WFH1vbwx7rsEtQcUXgf+uAwY+Sl 89gSP43of0qmLW39cNcHytq7iMqwJu2VbjRpkXO/wwGN6pdVxj6JMdWOE2YBHH6Y3zZt JYow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+NdinlcwIipK2AxQ+52YkcDL3y172tlBEWhb8NtIA2M=; b=M7s+p+28rREZyV62bqvB7KzmLBWpelUd1fKXTr6Szlhc9FP0xwgajQgPLX/n01tZYv 8iiS3QcsegorzwfsgNi3i5qosvL9GRQMSdVI06EaK49+IpyF97FHvM/aANwl2yy1i26r bxzfgdUBdmrEPHstfxq5V/Qv8qfDXVDZiB9R9a+PBaaF2T0ZLKHXH16L1/FXal1v+zBK WN2aHWlOSCr6QSIDZZgnlhqXuCtzmqHLWzKJKtDkBcMV+xELUdTdoobbiLv5CUKBXw52 YBUA7Ej6ENexeJZeciSXJRJct8usnfiJCLlQb0F5OQRIIC4qhX+hrMVun4DliNeFvYOA SfyQ== X-Gm-Message-State: AKaTC02Na0oTZwv/EOW2yjy7vRzS0nMobWw+qS3bf12Qml5uz9Rq3yf6fs+W3A+lGj9e4aBGniNWixP2ncQ8OA== X-Received: by 10.157.11.72 with SMTP id p8mr53713588otd.146.1481638194638; Tue, 13 Dec 2016 06:09:54 -0800 (PST) MIME-Version: 1.0 Received: by 10.182.213.229 with HTTP; Tue, 13 Dec 2016 06:09:54 -0800 (PST) In-Reply-To: References: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> <20161212054233.GU8460@kduck.kaduk.org> <01000158f1abc081-d4eddc58-3b4b-41dd-aa1e-0157d2fad812-000000@email.amazonses.com> <20161212163603.GV8460@kduck.kaduk.org> From: Doug Rabson Date: Tue, 13 Dec 2016 14:09:54 +0000 Message-ID: Subject: Re: ESTALE after cwd deleted by same NFS client To: Rick Macklem Cc: Benjamin Kaduk , Colin Percival , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 14:09:55 -0000 OPEN4_RESULT_PRESERVE_UNLINKED is described in the NFSv4.1 RFC On 13 December 2016 at 13:04, Rick Macklem wrote: > Benjamin Kaduk wrote: > > >On Mon, Dec 12, 2016 at 06:15:14AM +0000, Colin Percival wrote: > >> On 12/11/16 21:42, Benjamin Kaduk wrote: > >> > On Sun, Dec 11, 2016 at 11:06:42PM +0000, Colin Percival wrote: > >> >> If I run the following with /nfs/ being an NFS mount: > >> >> # mkdir /nfs/foo > >> >> # touch /nfs/foo/bar > >> >> # cd /nfs/foo > >> >> # rm -r /nfs/foo > >> >> # rm bar > If this is happening on a single client, something is broken. > - For this step, the name cache should have a miss and the Lookup should > fail with > ENOENT. I think the most likely failures are... > --> Either the name cache in the client is broken and has a hit > OR > --> The server replies NFS_OK for the lookup, even though it should have > been removed. > > When I get home on the weekend, I'll try this against a FreeBSD and Linux > server, to see > if it fails. If it does, then I'll try to figure out why. > (Version of NFS shouldn't matter for this case. They recently added the > optional case > of a server that can retain files after an NFSv4 Remove until the NFSv4 > Close, making > it more POSIX like. I think that's NFSv4.2, but it might be in NFSV4.1.) > > Did you try: > sysctl debug.disable_vnfullpath=1 (or something close to > disable_vnfullpath) > to see if it helped? > > rick > > >> >> > >> >> Then the final 'rm bar' fails with 'Stale NFS file handle'. > >> > > >> > Amusingly, this just came up recently: > >> > > >> > https://www.ietf.org/mail-archive/web/nfsv4/current/msg15115.html > (et seq) > >> > > >> > But I guess you did not specify which version of the NFS protocol you > were > >> > using... > >> > >> I'm using NFSv4.1, but this isn't quite the same... that link refers to > having > >> one NFS client remove a file out from underneath a different client, > while I'm > >> talking about having an NFS client remove a file from underneath > *itself*. > > > >I thought the reply mentioned (in passing) the case of a client removing a > >file out from under itself as well. But, maybe I was reading too fast. > > > >-Ben > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Wed Dec 14 00:34:07 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0D84C743AA; Wed, 14 Dec 2016 00:34:07 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-wj0-x230.google.com (mail-wj0-x230.google.com [IPv6:2a00:1450:400c:c01::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 86FF31C84; Wed, 14 Dec 2016 00:34:07 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-wj0-x230.google.com with SMTP id xy5so6001829wjc.0; Tue, 13 Dec 2016 16:34:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=cLJHsTtdEqaW3+fNMrFHwZ1gNVoD5TT9QA3HXXnwFMw=; b=dOwJdcb1CedKBW9Ea/w05pwZ8jeAw+aO4UlXN3foFQBNxmzR+BfYVISp+kD174/YwW C2KO3uF9fJN5ZkzB1+jnBLTSgww9arGNCRSkNa30hOLujxftvieMw2Ww6qBv7WqRBKKu 1jrn9bMgfpoNxWsOEIKK7FGO5Kbb6MGrHIV56YfNZQwZTXA/nSYU9jzQYDIhOW+0ot/S DzSnWVCSd0zg19xx+RmT4sHZzpBIAp1cEjxA3IGS9a1MafVpT3sJTtkRyrIKj9lcGWQB K559XDUrrm1W+5ewzI4fnxeySsDproRfmK2SCEZXrOUrsO5kP4X/gjHQUE7qKLI52eKj h9Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=cLJHsTtdEqaW3+fNMrFHwZ1gNVoD5TT9QA3HXXnwFMw=; b=SBnCsvUzcTp+hAUveyZR/iVjfrvJDj2Cx/Aiu6x6ECXFiQFZz6lW1hZxFpEYL2jqQb btSRBFXnF/wWYYquDUddh7ZVbhp8qcYl9Zd7w1X4U5v/zZNI7NgOw+AaXnriD9vKH23T c4pC3iRZ6hoF/Oh1JYP3r9v0IDrtjrHfFtA+S3WR4V4NiEm73Q+v5LFsexub0YCmXsgb tjcKH6OjpJgrA0343mbNczymyzy1ERELjHi6oZs+Mfu3NbjE4xbpjfoxyLstEMv4Zv2x U/dlCp7mq2xqqdC5BqfpeyZgqUHaY2ofbXbNchlwX2rh/p/gTAEzeEcszTNCJiOYttc6 teRw== X-Gm-Message-State: AKaTC02+jagxljIXaeH34WthXYU29eXTugjCR/FCAMzVBqLJmojQDbTm4kfReTdXRgvTj2pvOS6+FZPlds2AtA== X-Received: by 10.28.29.23 with SMTP id d23mr4711170wmd.91.1481675645237; Tue, 13 Dec 2016 16:34:05 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.169.195 with HTTP; Tue, 13 Dec 2016 16:34:04 -0800 (PST) In-Reply-To: References: From: Adam Vande More Date: Tue, 13 Dec 2016 18:34:04 -0600 Message-ID: Subject: Re: Re-sparse a file-backed IO device + zfs To: javocado Cc: FreeBSD virtualization , FreeBSD Questions , FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 00:34:08 -0000 On Mon, Dec 12, 2016 at 7:20 PM, javocado wrote: > Hi, > > I'm setting up a bhyve wherein: > > host # truncate -s 1T vol.file > host # du -ah vol.file > 200K vol.file > > host # /usr/sbin/bhyve ... -s 4,ahci-hd,vol.file ... > > Then inside the bhyve I create a zpool (ada0 = vol.file): > > bhyve # zpool create -O devices=off -O atime=off -O compression=on -m > /mnt/data1 data1 ada0 > I think there used to be a utility called sparsify. -- Adam From owner-freebsd-fs@freebsd.org Wed Dec 14 07:35:36 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9AC3DC764F9 for ; Wed, 14 Dec 2016 07:35:36 +0000 (UTC) (envelope-from 01000158fc3d9bab-65b7741f-ca02-4024-a34c-b8be22a3b5e6-000000@amazonses.com) Received: from a8-26.smtp-out.amazonses.com (a8-26.smtp-out.amazonses.com [54.240.8.26]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E93C14FF for ; Wed, 14 Dec 2016 07:35:35 +0000 (UTC) (envelope-from 01000158fc3d9bab-65b7741f-ca02-4024-a34c-b8be22a3b5e6-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57; d=tarsnap.com; t=1481700645; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=BewlLjArjIK7Pujs1R1E/u88FJlmXL6L6F6nSLebajA=; b=J+lTPRVTic+dGLVdOTFQJMPykGiac7/anjnbVk53iA29d/Fu3nYT19ilfx6uSNS+ d8rIpK2JuX/H/Zx51HJ6ojog1J3kX/CFZtkOE/PAHJOE6+TfadAEAC+jZg2lmb349j+ T4vVSlvqLJZCIrboQyVvrL4PnfK6Tw1gyo9ma/78= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1481700645; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=BewlLjArjIK7Pujs1R1E/u88FJlmXL6L6F6nSLebajA=; b=uOXwv1KBvRaozLPHSrVrIbIeFOIMZQn/QksCXkQPywEmBvPpA/EWf0lgcVsa/Qic 7bJLNCLYgd18mE749GBcEvRD9CDP9gV28MbbWPhCwYsdOpQhBiOhrodG6KqnuE9pmFz nL5IRdeUD02BMgwjDoBPBg4IUtx/DqH63nFhGM4E= Subject: Re: ESTALE after cwd deleted by same NFS client To: Rick Macklem , Benjamin Kaduk References: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> <20161212054233.GU8460@kduck.kaduk.org> <01000158f1abc081-d4eddc58-3b4b-41dd-aa1e-0157d2fad812-000000@email.amazonses.com> <20161212163603.GV8460@kduck.kaduk.org> Cc: "freebsd-fs@freebsd.org" From: Colin Percival Message-ID: <01000158fc3d9bab-65b7741f-ca02-4024-a34c-b8be22a3b5e6-000000@email.amazonses.com> Date: Wed, 14 Dec 2016 07:30:45 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2016.12.14-54.240.8.26 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 07:35:36 -0000 On 12/13/16 05:04, Rick Macklem wrote: >>>> On Sun, Dec 11, 2016 at 11:06:42PM +0000, Colin Percival wrote: >>>>> If I run the following with /nfs/ being an NFS mount: >>>>> # mkdir /nfs/foo >>>>> # touch /nfs/foo/bar >>>>> # cd /nfs/foo >>>>> # rm -r /nfs/foo >>>>> # rm bar > If this is happening on a single client Yes. At the time I ran this, there was only one client. > something is broken. > - For this step, the name cache should have a miss and the Lookup should fail with > ENOENT. I think the most likely failures are... > --> Either the name cache in the client is broken and has a hit > OR > --> The server replies NFS_OK for the lookup, even though it should have been removed. The server replied ESTALE for the 'rm bar'. Is that the lookup you mean? I don't think a lookup happened for /nfs/foo after the 'rm -r' completed, since it was already the current directory (despite no longer existing). > Did you try: > sysctl debug.disable_vnfullpath=1 (or something close to disable_vnfullpath) > to see if it helped? If you mean debug.disablefullpath=1, yes, I tried that. I also tried setting vfs.lookup_shared=0; also no change. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-fs@freebsd.org Wed Dec 14 09:57:33 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1179BC76C0F for ; Wed, 14 Dec 2016 09:57:33 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C7F701C9F for ; Wed, 14 Dec 2016 09:57:32 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 4395C28437; Wed, 14 Dec 2016 10:57:24 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id A1E9128429; Wed, 14 Dec 2016 10:57:23 +0100 (CET) Subject: Re: Re-sparse a file-backed IO device + zfs To: Jan Bramkamp , freebsd-fs@freebsd.org References: <43b81c7e-8a7b-73f4-0a13-5c91c92663b6@rlwinm.de> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <58511783.9000702@quip.cz> Date: Wed, 14 Dec 2016 10:57:23 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 In-Reply-To: <43b81c7e-8a7b-73f4-0a13-5c91c92663b6@rlwinm.de> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 09:57:33 -0000 Jan Bramkamp wrote on 2016/12/13 10:59: > On 13/12/2016 07:54, Karli Sjöberg wrote: >> Wouldn't you have needed TRIM on the disk(s) inside the VM and it >> would have taken care of all this for you? > > The ahci-hd driver supports TRIM on ZFS volumes. Why do you use sparse > file instead of a ZFS volume? I something had not changed you need twice the space of ZVOL to make a snapshot. So in some scenarios it is better to use files instead of ZVOL. Miroslav Lachman From owner-freebsd-fs@freebsd.org Wed Dec 14 17:41:48 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9815C80582 for ; Wed, 14 Dec 2016 17:41:48 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0071.outbound.protection.outlook.com [65.55.169.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7667A2D9 for ; Wed, 14 Dec 2016 17:41:47 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0192.CANPRD01.PROD.OUTLOOK.COM (10.165.218.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.761.9; Wed, 14 Dec 2016 13:09:02 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.0771.014; Wed, 14 Dec 2016 13:09:02 +0000 From: Rick Macklem To: Colin Percival , Benjamin Kaduk CC: "freebsd-fs@freebsd.org" Subject: Re: ESTALE after cwd deleted by same NFS client Thread-Topic: ESTALE after cwd deleted by same NFS client Thread-Index: AQHSVANFVKFixagVOUGZ8mLDXQwbl6EDzQuAgAC4aM2AAVLGrIABN7mAgABdEGA= Date: Wed, 14 Dec 2016 13:09:02 +0000 Message-ID: References: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> <20161212054233.GU8460@kduck.kaduk.org> <01000158f1abc081-d4eddc58-3b4b-41dd-aa1e-0157d2fad812-000000@email.amazonses.com> <20161212163603.GV8460@kduck.kaduk.org> , <01000158fc3da2c5-c13da088-e7b9-4ac0-ac01-ec49a275dd24-000000@email.amazonses.com> In-Reply-To: <01000158fc3da2c5-c13da088-e7b9-4ac0-ac01-ec49a275dd24-000000@email.amazonses.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: ea3d2910-674b-4183-c401-08d42422603c x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YTXPR01MB0192; x-microsoft-exchange-diagnostics: 1; YTXPR01MB0192; 7:bRHtHvY8CNLGWRkHmNVAAZdMmrgUKkaAoCy4MjPMfK+V0Z0N4JXyXuq3hxRvVk3cjcqfhq8DKghzIPsEInPaSoI6Cj68KBkmbb82UaK0fbjtsZbhIdDq5WONFAmlwbGs9ChhCSRh7l14JsFlyQtpsoNDiluiAtk93H0q1iwjkbksZeCpNcI03A/yUywReogW4o7HSqsccsMiCaJX80QvWgX7eeW4noPOXWEpLF0n+QYNsSEnO+sA0iuJ59EtbMZVMsnDz+Md093McTJYg4PetxVLj9XJ0WMW7IuVBXu4PMny9yqmfXPWJt+ZftecT1awdcCCpKnPCjVb8cotlL3CzrTevzY8b8d1QZIif4jlD5oz+lp0KKCn+4in64UwLNFNu9yLNH6miuHdfkYR5QDxi9TYpUZMlVg7/seUpa8bA736Z6NbDd2OJaHi4Gfaq4DxbQKk5ChpwhEt3ZAhuhOc+w== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6041248)(20161123555025)(20161123560025)(20161123562025)(20161123564025)(6072148); SRVR:YTXPR01MB0192; BCL:0; PCL:0; RULEID:; SRVR:YTXPR01MB0192; x-forefront-prvs: 01565FED4C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(39450400003)(199003)(24454002)(189002)(74316002)(102836003)(5660300001)(2171001)(305945005)(101416001)(68736007)(86362001)(6506006)(9686002)(6436002)(93886004)(229853002)(33656002)(77096006)(106116001)(50986999)(54356999)(81156014)(76176999)(8936002)(97736004)(5001770100001)(92566002)(38730400001)(105586002)(81166006)(74482002)(2950100002)(2906002)(8676002)(106356001)(122556002)(189998001)(2900100001)(4326007)(3660700001)(3280700002)(7696004); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0192; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Dec 2016 13:09:02.7385 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0192 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 17:41:48 -0000 Colin Percival wrote: >On 12/13/16 05:04, Rick Macklem wrote: >>>>> On Sun, Dec 11, 2016 at 11:06:42PM +0000, Colin Percival wrote: >>>>>> If I run the following with /nfs/ being an NFS mount: >>>>>> # mkdir /nfs/foo >>>>>> # touch /nfs/foo/bar >>>>>> # cd /nfs/foo >>>>>> # rm -r /nfs/foo >>>>>> # rm bar >> If this is happening on a single client > >Yes. At the time I ran this, there was only one client. > >> something is broken. >> - For this step, the name cache should have a miss and the Lookup should= fail with >> ENOENT. I think the most likely failures are... >> --> Either the name cache in the client is broken and has a hit >> OR >> --> The server replies NFS_OK for the lookup, even though it should ha= ve been >removed. > >The server replied ESTALE for the 'rm bar'. Is that the lookup you mean? = I >don't think a lookup happened for /nfs/foo after the 'rm -r' completed, si= nce >it was already the current directory (despite no longer existing). If there was no Lookup between "rm -r /nfs/foo" and "rm bar" then that is t= he problem. The NFS client remove makes a cache call (can't remember the exact function name without looking at the code) which should invalidate the mapp= ing of the name ("bar" in this case). Then, a Lookup of "bar" should occur. --> I'll try and reproduce this when I get home on the weekend. rick [stuff snipped] From owner-freebsd-fs@freebsd.org Fri Dec 16 02:51:05 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55AA6C77CB4 for ; Fri, 16 Dec 2016 02:51:05 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0065.outbound.protection.outlook.com [104.47.42.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F368E1C5C for ; Fri, 16 Dec 2016 02:51:04 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.789.14; Thu, 15 Dec 2016 23:17:46 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.0789.014; Thu, 15 Dec 2016 23:17:46 +0000 From: Rick Macklem To: Colin Percival , Benjamin Kaduk CC: "freebsd-fs@freebsd.org" Subject: Re: ESTALE after cwd deleted by same NFS client Thread-Topic: ESTALE after cwd deleted by same NFS client Thread-Index: AQHSVANFVKFixagVOUGZ8mLDXQwbl6EDzQuAgAC4aM2AAVLGrIABN7mAgABdEGCAAjtv0g== Date: Thu, 15 Dec 2016 23:17:46 +0000 Message-ID: References: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> <20161212054233.GU8460@kduck.kaduk.org> <01000158f1abc081-d4eddc58-3b4b-41dd-aa1e-0157d2fad812-000000@email.amazonses.com> <20161212163603.GV8460@kduck.kaduk.org> , <01000158fc3da2c5-c13da088-e7b9-4ac0-ac01-ec49a275dd24-000000@email.amazonses.com>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: 088d3dc4-6b95-4984-7716-08d42540944f x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YTXPR01MB0189; x-microsoft-exchange-diagnostics: 1; YTXPR01MB0189; 7:MVlXM1QZKPs3H33quIC7/LSATLXruLx1Ueel2crflOtgMZYbLadDTZfVBb+YLLYhC5oqL6b/W8NrKivqEtszcZIPntzVu98Oh4G43Nkdk1443PZa9M3kznNfJF2DvblZbR210kJWyDsmgMjBaHa8nYuyMBTyMGPZR+HMpaHQqhSGTTFasVzG/umFqRz35ufzd+Oj9hWXx9i+2MULgKVFxcbKsOVECpF0U/E26x3Z/nmo1dqaCs0lqFgdtX1h8sletQ5SMjG0COqZP7oWx8yg1K5VHIYPsb4pgjgugg0P2C3wXEY7qMNIH16ToWOkRLWisB3srI9HQ1HPkAPkkAGYNcZG064Sx7ChB+3IeHoi9zVCmkHNFhgKSDSh0mxVdI1e7YuBC4nb0deGg3HzacWjWUr7CYV+T7pAOYiAYv6UTjRMN2EvyASbbfJkVQuc7PFJbPPaI4Hd1wYo6gjzogXA7Q== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123560025)(20161123555025)(20161123562025)(20161123564025)(6072148); SRVR:YTXPR01MB0189; BCL:0; PCL:0; RULEID:; SRVR:YTXPR01MB0189; x-forefront-prvs: 0157DEB61B x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(39450400003)(189002)(199003)(24454002)(106356001)(68736007)(5660300001)(4326007)(8676002)(74482002)(9686002)(2906002)(8936002)(92566002)(2900100001)(81166006)(3280700002)(189998001)(122556002)(101416001)(33656002)(3660700001)(305945005)(97736004)(54356999)(5001770100001)(76176999)(50986999)(93886004)(102836003)(81156014)(2171001)(2950100002)(77096006)(74316002)(105586002)(7696004)(86362001)(106116001)(229853002)(38730400001)(6436002)(6506006); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0189; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Dec 2016 23:17:46.1202 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0189 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 02:51:05 -0000 Colin Percival wrote: >On 12/13/16 05:04, Rick Macklem wrote: >>>>> On Sun, Dec 11, 2016 at 11:06:42PM +0000, Colin Percival wrote: >>>>>> If I run the following with /nfs/ being an NFS mount: >>>>>> # mkdir /nfs/foo >>>>>> # touch /nfs/foo/bar >>>>>> # cd /nfs/foo >>>>>> # rm -r /nfs/foo >>>>>> # rm bar >> If this is happening on a single client > >Yes. At the time I ran this, there was only one client. > I will try and play with this when I get home, but you could test the follo= wing simple change: (I'll just explain the patch, since I don't have sources han= dy) In sys/fs/nfsclient/nfs_clvnops.c: Line #s 1678, 1679 (for remove) and 2180, 2181 (for rmdir) look like: if (error =3D=3D ENOENT) error =3D 0; This is done since, if it doesn't exist, it has been "removed". It seems to me that ESTALE means it has been removed too. (Actually the directory the name was in has been removed, but that implies = that the name is gone too.) You could try changing the code at 1678 and 2180 to: if (error =3D=3D ENOENT || error =3D=3D ESTALE) error =3D 0; and see if that "fixes" the problem. (This hack has existed in the code for= a long time and changing it to also do ESTALE seems reasonable to me.) If this doesn't fix it, then it might be "do sillyrename for directories", = but that is a more significant (I might call it scarier) change. Have fun with it, rick From owner-freebsd-fs@freebsd.org Fri Dec 16 12:16:21 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BE59C82E5F for ; Fri, 16 Dec 2016 12:16:21 +0000 (UTC) (envelope-from admin@x145.save85off.com) Received: from x145.save85off.com (x145.save85off.com [43.240.238.145]) by mx1.freebsd.org (Postfix) with ESMTP id 183371D66 for ; Fri, 16 Dec 2016 12:16:20 +0000 (UTC) (envelope-from admin@x145.save85off.com) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=save85off; d=x145.save85off.com; h=MIME-Version:From:To:Date:Subject:Content-Type:Content-Transfer-Encoding; i=admin@x145.save85off.com; bh=fwkmsJdIUnWTJ0n9TubdI0x+RgI=; b=ZoimcSF6nGyF6x0fMl2TicF+cvTqVCDasKlwpkTe9wF3/YTg8pQWlXe1aoWtk4XvS88TnmlDKAVm +cs1qdR71ERztYp9/h9Zzgmsm1gPTFNWA0oCHE4/gSzdg65gccVMq4IGw6Xi9rWwgutaprhQ4ESM z6iPITi5uXFAGomWPFU= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=save85off; d=x145.save85off.com; b=fxbph0p5QV9AO4u5TJOR7dtugKF6J4vmjfcn+z1+QEHOrLbn76BYZK6hNefsDTy1/u1esfocLHbf A7UA6WrvT5ySWytw7qNzvOGdGXZx6AksIkNYBHSD9k0khNlaAU/kHyTZVObbBw1OGESQysBoS/Mm cDl8Dauq2g6EAnC2yQ4=; From: "UGG Big Deals" To: freebsd-fs@freebsd.org Date: 16 Dec 2016 20:04:11 +0800 Subject: Big Sales are the reason for the season win 140$ MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 12:16:21 -0000 From owner-freebsd-fs@freebsd.org Fri Dec 16 20:15:02 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ADB78C83B02 for ; Fri, 16 Dec 2016 20:15:02 +0000 (UTC) (envelope-from 010001590945e9b3-015a4d05-2646-44ba-9db9-415e8b9119dd-000000@amazonses.com) Received: from a8-176.smtp-out.amazonses.com (a8-176.smtp-out.amazonses.com [54.240.8.176]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6B53E927 for ; Fri, 16 Dec 2016 20:15:02 +0000 (UTC) (envelope-from 010001590945e9b3-015a4d05-2646-44ba-9db9-415e8b9119dd-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57; d=tarsnap.com; t=1481919293; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=U5WpkV6CS3uFMsOMwY+QsfSQVSkdfgUWB+6nA7gX6W0=; b=Mo3uWstTHnpYIROr8FFvTS/iYpBqngppb/fOgvt9QdDkAfobiWo6q1LlkTY6dAvu 704QiAGptc2YL3O5p2ehuAR5+XgLB4O3B8JwqFevlettv3PU7qiuEdQZ9zUVa8OtDRt 9+EdlrkR9gSPLvlwG6+cA8cXN4Zx9GQuraEcnmhg= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1481919293; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=U5WpkV6CS3uFMsOMwY+QsfSQVSkdfgUWB+6nA7gX6W0=; b=atV+EKuGgaP0yp8YLVs06uGNeRGKXcP0H/j0Bn18eYTFwm7hC5b+L/ZKMtBt0+/r 2y929H7j8vybv/Fi+tXGxNakTtRVO6NG8DBwYozZxBYmU7qyUGg6RDNcNQXGmokPls0 EezgLg+nOL8bGqNSDFvy1bG8CBIy8upHXDuP5auM= Subject: Re: ESTALE after cwd deleted by same NFS client To: Rick Macklem References: <01000158f023675b-41b35a73-4428-4937-853b-62db4fb9b984-000000@email.amazonses.com> <20161212054233.GU8460@kduck.kaduk.org> <01000158f1abc081-d4eddc58-3b4b-41dd-aa1e-0157d2fad812-000000@email.amazonses.com> <20161212163603.GV8460@kduck.kaduk.org> <01000158fc3da2c5-c13da088-e7b9-4ac0-ac01-ec49a275dd24-000000@email.amazonses.com> Cc: "freebsd-fs@freebsd.org" From: Colin Percival Message-ID: <010001590945e9b3-015a4d05-2646-44ba-9db9-415e8b9119dd-000000@email.amazonses.com> Date: Fri, 16 Dec 2016 20:14:53 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2016.12.16-54.240.8.176 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 20:15:02 -0000 On 12/15/16 15:17, Rick Macklem wrote: >>>>>> On Sun, Dec 11, 2016 at 11:06:42PM +0000, Colin Percival wrote: >>>>>>> If I run the following with /nfs/ being an NFS mount: >>>>>>> # mkdir /nfs/foo >>>>>>> # touch /nfs/foo/bar >>>>>>> # cd /nfs/foo >>>>>>> # rm -r /nfs/foo >>>>>>> # rm bar > > I will try and play with this when I get home, but you could test the following > simple change: (I'll just explain the patch, since I don't have sources handy) > In sys/fs/nfsclient/nfs_clvnops.c: > Line #s 1678, 1679 (for remove) and 2180, 2181 (for rmdir) look like: > if (error == ENOENT) > error = 0; > This is done since, if it doesn't exist, it has been "removed". > It seems to me that ESTALE means it has been removed too. > (Actually the directory the name was in has been removed, but that implies that > the name is gone too.) This doesn't fix the problem. This particular ESTALE is coming earlier, from nfsrpc_lookup, when we attempt to look up a path relative to a directory which no longer exists (and thus through a file handle which is stale). Interestingly, making this change in nfs_lookup > --- sys/fs/nfsclient/nfs_clvnops.c (revision 310132) > +++ sys/fs/nfsclient/nfs_clvnops.c (working copy) > @@ -1144,7 +1144,7 @@ > *vpp = NULLVP; > } > > - if (error != ENOENT) { > + if (error != ENOENT && error != ESTALE) { > if (NFS_ISV4(dvp)) > error = nfscl_maperr(td, error, (uid_t)0, > (gid_t)0); fixes the case I described above (for some definition of "fixes" -- I'm not sure if this is the correct way of handling ESTALE here?) but I'm still seeing ESTALEs from buildworld's cleandir so I think there must be some other place where something odd is happening. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid