From owner-freebsd-fs@freebsd.org Thu Sep 12 14:41:41 2019 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9C787F7DA7 for ; Thu, 12 Sep 2019 14:41:41 +0000 (UTC) (envelope-from daveb@spectralogic.com) Received: from us-smtp-delivery-147.mimecast.com (us-smtp-delivery-147.mimecast.com [216.205.24.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.mimecast.com", Issuer "DigiCert Global CA G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46ThKw490Dz4LKq for ; Thu, 12 Sep 2019 14:41:40 +0000 (UTC) (envelope-from daveb@spectralogic.com) Received: from isotope.sldomain.com (192.30.190.20 [192.30.190.20]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-299-bkPFyI3vMQ6ITU4uAnUIwA-1; Thu, 12 Sep 2019 10:41:38 -0400 Received: from isotope.sldomain.com (192.168.200.20) by isotope.sldomain.com (192.168.200.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 12 Sep 2019 08:41:37 -0600 Received: from isotope.sldomain.com ([fe80::5567:5b08:1164:d2ee]) by isotope.sldomain.com ([fe80::5567:5b08:1164:d2ee%6]) with mapi id 15.01.1713.004; Thu, 12 Sep 2019 08:41:37 -0600 From: Dave Baukus To: "freebsd-fs@freebsd.org" Subject: ZFS: panic in vdev_dtl_reassess() Thread-Topic: panic in vdev_dtl_reassess() Thread-Index: AQHVaXguxEEqRa6HcUaBd8g/k/Izvw== Date: Thu, 12 Sep 2019 14:41:37 +0000 Message-ID: <1cd944ef-c5f8-1ae7-dd0b-7e068ab21cb4@spectralogic.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: MIME-Version: 1.0 X-MC-Unique: bkPFyI3vMQ6ITU4uAnUIwA-1 X-Mimecast-Spam-Score: 0 X-Rspamd-Queue-Id: 46ThKw490Dz4LKq X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of daveb@spectralogic.com designates 216.205.24.147 as permitted sender) smtp.mailfrom=daveb@spectralogic.com X-Spamd-Result: default: False [-2.50 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.991,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:216.205.24.0/24]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[spectralogic.com]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-0.21)[ipnet: 216.205.24.0/24(-0.47), asn: 30031(-0.51), country: US(-0.05)]; MIME_BASE64_TEXT(0.10)[]; TO_DN_EQ_ADDR_ALL(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[147.24.205.216.list.dnswl.org : 127.0.3.1]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:30031, ipnet:216.205.24.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Sep 2019 14:41:41 -0000 T24gRnJlZUJTRCAxMi4wLVNUQUJMRSBJIGhhdmUgYSBwYW5pYyBpbiB2ZGV2X2R0bF9yZWFzc2Vz cygpIGJlY2F1c2Ugc3BhLT5zcGFfZHNsX3Bvb2wtPmRwX3NjYW4gaXMgTlVMTA0KDQojNyAgdHJh cCAoZnJhbWU9MHhmZmZmZmUwMjZkNmUwN2QwKSBhdCBzeXMvYW1kNjQvYW1kNjQvdHJhcC5jOjQ0 Mw0KIzggIDxzaWduYWwgaGFuZGxlciBjYWxsZWQ+DQojOSAgdmRldl9kdGxfcmVhc3Nlc3MgKHZk PTB4ZmZmZmY4MDAxMDY2MTAwMCwgdHhnPTAsIHNjcnViX3R4Zz0wLCBzY3J1Yl9kb25lPTApIGF0 IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvdmRldi5jOjI1 MTkNCiMxMCB2ZGV2X2R0bF9yZWFzc2VzcyAodmQ9MHhmZmZmZjgwMDFjNmQ3MDAwLCB0eGc9MCwg c2NydWJfdHhnPTAsIHNjcnViX2RvbmU9MCkgYXQgc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJp cy91dHMvY29tbW9uL2ZzL3pmcy92ZGV2LmM6MjUxMg0KIzExIHZkZXZfZHRsX3JlYXNzZXNzICh2 ZD0weGZmZmZmODAzMThkZjUwMDAsIHR4Zz0wLCBzY3J1Yl90eGc9MCwgc2NydWJfZG9uZT0wKSBh dCBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3ZkZXYuYzoy NTEyDQojMTIgc3BhX3ZkZXZfc3RhdGVfZXhpdCAoc3BhPTB4ZmZmZmZlMDFiZDdmZTAwMCwgdmQ9 MHgwLCBlcnJvcj0wKSBhdCBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24v ZnMvemZzL3NwYV9taXNjLmM6MTM5Ng0KIzEzIHNwYV9hc3luY190aHJlYWRfdmQgKGFyZz0weGZm ZmZmZTAxYmQ3ZmUwMDApIGF0IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1v bi9mcy96ZnMvc3BhLmM6NzMyMA0KDQpsaW5lIDI1MTkgb2YgdmRldi5jIGlzOg0KaWYgKHZkLT52 ZGV2X29wcy0+dmRldl9vcF9sZWFmKSB7DQogICAgIGRzbF9zY2FuX3QgKnNjbiA9IHNwYS0+c3Bh X2RzbF9wb29sLT5kcF9zY2FuOw0KICAgICAuLi4NCg0KSSBiZWxpZXZlIEkga25vdyBob3cgSSBn b3QgaGVyZSwgYnV0IHRoZSBwcm9wZXIgc29sdXRpb24gaXMgbm90IG9idmlvdXMgdG8gbWUuDQpJ IGhhdmUgYSAzIHBhcnQgdHJhaW4gd3JlY2s6DQoNClBhcnQxOg0KL3Vzci90ZXN0cy9zeXMvZGV2 YWQvZGV2YWRfdGVzdCBpcyBleGVjdXRpbmcgdHVybmluZyBkaXNrIFBIWXMgb24gYW5kIG9mZi4N Cg0KUGFydDI6DQpBIHByb2Nlc3MgaXMgZXhlY3V0aW5nIHpwb29sIGltcG9ydCAobm8gYXJndW1l bnRzKTsgdGhpcyBsZWFkcyB0bw0Kc3BhX3RyeWltcG9ydCgpLiBJIGtub3cgc3BhX2xvYWQoKSBm YWlsZWQgYmVjYXVzZSBzcGEtPnNwYV9sb2FkX3N0YXRlID09IFNQQV9MT0FEX0VSUk9SDQphbmQg SSBjb25jbHVkZSB0aGF0IHRoZSByb290IGZhaWx1cmUgb2NjdXJyZWQgaW4gZHNsX3Bvb2xfaW5p dCgpIGFzIGNhbGxlZA0KZnJvbSBzcGFfbGRfb3Blbl9yb290YnAoKSBiZWNhdXNlIHNwYS0+c3Bh X2RzbF9wb29sIGFuZCBzcGEtPnNwYV9tZXRhX29ianNldA0KYXJlIGJvdGggTlVMTC4gVGhpcyBz cGFfdHJ5aW1wb3J0KCkgdGhyZWFkIGlzIHdhaXRpbmcgZm9yIHRoZSAgU0NMX0NPTkZJRyB8IFND TF9TVEFURSBsb2NrcyBoZWxkDQoNCmJ5IHRoZSB0aHJlYWQgaW4gUGFydDM6DQoNCiMzICBfY3Zf d2FpdCAoY3ZwPTB4ZmZmZmZlMDFiZDdmZWY0OCwgbG9jaz0weGZmZmZmZTAxYmQ3ZmVmMTgpIGF0 IHN5cy9rZXJuL2tlcm5fY29uZHZhci5jOjE0Ng0KIzQgIHNwYV9jb25maWdfZW50ZXIgKHNwYT0w eGZmZmZmZTAxYmQ3ZmUwMDAsIGxvY2tzPTMsIHRhZz08b3B0aW1pemVkIG91dD4sIHJ3PVJXX1JF QURFUikgYXQgc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9z cGFfbWlzYy5jOjU1OQ0KIzUgIHNwYV9jb25maWdfZ2VuZXJhdGUgKHNwYT0weGZmZmZmZTAxYmQ3 ZmUwMDAsIHZkPTB4ZmZmZmY4MDMxOGRmNTAwMCwgdHhnPTE4NDQ2NzQ0MDczNzA5NTUxNjE1LCBn ZXRzdGF0cz0xKSBhdCBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMv emZzL3NwYV9jb25maWcuYzo0MTUNCiM2ICBzcGFfdHJ5aW1wb3J0ICh0cnljb25maWc9PG9wdGlt aXplZCBvdXQ+KSBhdCBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMv emZzL3NwYS5jOjU2NDINCiM3ICB6ZnNfaW9jX3Bvb2xfdHJ5aW1wb3J0ICh6Yz0weGZmZmZmZTAy MzMxZmMwMDApIGF0IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96 ZnMvemZzX2lvY3RsLmM6MTc1Ng0KIzggIHpmc2Rldl9pb2N0bCAoZGV2PTxvcHRpbWl6ZWQgb3V0 PiwgemNtZD08b3B0aW1pemVkIG91dD4sIGFyZz0weGZmZmZmZTAwZmY0MjM4YzAgIlxhIiwgZmxh Zz08b3B0aW1pemVkIG91dD4sIHRkPTxvcHRpbWl6ZWQgb3V0PikgYXQgc3lzL2NkZGwvY29udHJp Yi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfaW9jdGwuYzo2ODA0DQoNClBhcnQz Og0KVGhlIHNwYV9hc3luY190aHJlYWRfdmQoKSB0aHJlYWQgaXMgcHJvY2Vzc2luZyBhICh0YXNr cyAmIFNQQV9BU1lOQ19SRU1PVkUpIGV2ZW50IGZvcg0KdGhlIHNhbWUgc3BhIHRoYXQgc3BhX3Ry eWltcG9ydCgpIHdhaXRpbmcgdG8gY29tcGxldGUuIFRoZSBzcGFfYXN5bmNfdGhyZWFkX3ZkKCkg dGhyZWFkDQoNCmlzIFNDTF9TVEFURV9BTEwgbG9ja3MgYmxvY2tpbmcgdGhlIHNwYV90cnlpbXBv cnQoKSB0aHJlYWQuDQoNClRodXMsIEkgaGF2ZSB0aGUgc3BhX2FzeW5jX3RocmVhZF92ZCgpIHdh bGtpbmcgdGhyb3VnaCB0aGUgdmRldnMgb2YgYW4gZXBoZW1lcmFsIHNwYSB0aGF0IGlzDQpub3Qg ZnVsbHkgaW5pdGlhbGl6ZWQgYW5kIHdlIGNyYXNoIHdoZW4gc3BhX2FzeW5jX3RocmVhZF92ZCgp IGRlcmVmZXJlbmNlcyBzcGEtPnNwYV9kc2xfcG9vbC4NCg0KSXQgc2VlbXMgdGhhdCBhIHNwYSB0 aGF0IGlzIGJlaW5nIGNvbnN0cnVjdGVkIHNpbXBseSB0byBnbGVhbiB0aGUgcG9vbCdzIGNvbmZp Z3VyYXRpb24NCnNob3VsZCBiZSB0YWdnZWQgc28gdGhhdCBzcGFfYXN5bmNfdGhyZWFkX3ZkKCkg YW5kIG90aGVycyBkb24ndCBhc3N1bWUgaXQgaXMgZnVsbHkNCmNvbnN0cnVjdGVkLg0KDQotLQ0K RGF2ZSBCYXVrdXMNCg0K From owner-freebsd-fs@freebsd.org Fri Sep 13 15:18:09 2019 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 88C86F4020 for ; Fri, 13 Sep 2019 15:18:09 +0000 (UTC) (envelope-from mail@florian-schulze.net) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46VK5X33Mtz3Hy0 for ; Fri, 13 Sep 2019 15:18:07 +0000 (UTC) (envelope-from mail@florian-schulze.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 2DF4721EA0; Fri, 13 Sep 2019 11:18:07 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Fri, 13 Sep 2019 11:18:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= florian-schulze.net; h=from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-type :content-transfer-encoding; s=fm1; bh=j2NzawUwK4x/Zu7WojJQqlzs/U iYsbBgjVdyhDVwHm4=; b=Fn7nAUNSXgQcIVmRGu+3qsll591sAacZ1PUIdJNyMI DC4b16nCXZEkKZ45A7quGxw7EfbY6F1xPN2+SWzYUL0lF7CZUo2DFZfRpkUgFBqU D4IKBCH1imRfeY7sh/rhueGRPJBwtGy3seW91xcMhYm4xfKxw6pv9yXPST/noAH9 52uCLvBGGFY2nH9WoA3qI5hG2HUdwjcjMB8UarSJ3P/bx91pBQ1iSR+NsMldXVP8 uoAPH3MCzLH27YLuVw4XbkefjLuBB/FHZelgn3WhghRdDEyfnMpcIOm8nhIW8mo5 aq8SFgCKXgoM0GdUT/ahnnFYlPHMU2hwxC+cx++ZZHdA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=j2NzawUwK4x/Zu7WojJQqlzs/UiYsbBgjVdyhDVwH m4=; b=hyhHULh9T1eiCgrAKrvfDQwfDt9qQUAJkhw0EYDlXQWdTuR03Ce14YbKg B9xBdjbh5qtPSuU1awmSevPc1lmJONaNFUQjkWSJ12uOMm1supojJln4a8h/MT4A Ptw6bC0OBTi12bUE/EJ5ar+d8MLEjdFVXJkJANAilvqb945MIoJJUcPppHbe4puf ySX27GU00jAz/gucoQc/k+SxURKyGI+Lu0CJRbCAj1Yii4VqVTLMzalx8TLfExXH J2QIJN+fbAPanjlGDTBfbICblZ0I29TdbcG42BH/Z7Jt6ijbRz6Uca8KAmOGo/8h KWQnllxnk00wlyI0RuxJC5n5UdTEg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrtdejgdekiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffoffkjghfgggtgfesthhqmh dtredttdenucfhrhhomhepfdfhlhhorhhirghnucfutghhuhhliigvfdcuoehmrghilhes fhhlohhrihgrnhdqshgthhhulhiivgdrnhgvtheqnecuffhomhgrihhnpehgihhthhhusg drtghomhenucfkphepjeelrddvgeehrdehkedrvdeknecurfgrrhgrmhepmhgrihhlfhhr ohhmpehmrghilhesfhhlohhrihgrnhdqshgthhhulhiivgdrnhgvthenucevlhhushhtvg hrufhiiigvpedt X-ME-Proxy: Received: from [192.168.1.182] (p4ff53a1c.dip0.t-ipconnect.de [79.245.58.28]) by mail.messagingengine.com (Postfix) with ESMTPA id 3B13AD60063; Fri, 13 Sep 2019 11:18:06 -0400 (EDT) From: "Florian Schulze" To: "Konstantin Belousov" Cc: freebsd-fs@freebsd.org Subject: Re: held file reference issue with ZFS and nullfs Date: Fri, 13 Sep 2019 17:18:03 +0200 X-Mailer: MailMate (1.12.5r5635) Message-ID: <7CCB09C2-3A29-4AF7-8F7B-2834F4D1421C@florian-schulze.net> In-Reply-To: <5B1BEA13-F8CB-4E3C-9236-629892AF48F5@florian-schulze.net> References: <45B080A9-DE0F-4633-91F8-71438408D4B8@florian-schulze.net> <20190906124336.GE2559@kib.kiev.ua> <5B1BEA13-F8CB-4E3C-9236-629892AF48F5@florian-schulze.net> MIME-Version: 1.0 Content-Type: text/plain; format=flowed Content-Transfer-Encoding: quoted-printable Autocrypt: addr=mail@florian-schulze.net; keydata=mQENBFliSUoBCADLKPuy1qhTI+A3XRleyZ7WlXhRfYiXBCSMiIqUb+nx88xKLVAclRJqMnqlJHzAJcb4JW9i3riRU+V1rVG9BDaBCCih5Si0BQO5W84SpaadTdcE3rUO8Ihdxc6dJN9DiAtp26WyO1fXSA2NkwpakIerxlF91knkrnSa/VgLrlB+Ahtcisbsn9XYSnv0wr/njQJOfJMsmJfWAKXWWaWDWb0Z4/UITsBe259Ppkf6bTYS3j7oCPqygPzdyc14ILINb0N3ksT3tKLiS4b2gh7RAO4uf/C8TvlZSYvoX/sel5xFFxcgmqG+V0d4jjhdvY5zFHU+/Egr5y3HPgSQjevEHsNxABEBAAG0KkZsb3JpYW4gU2NodWx6ZSA8bWFpbEBmbG9yaWFuLXNjaHVsemUubmV0PokBOQQTAQgAIwUCWWJJSgIbAwcLCQgHAwIBBhUIAgkKCwQWAgMBAh4BAheAAAoJEC0N2jSLcB7qWo4H/jEgS6Id1TkRtlh+vZanpswS83sH8pOfc20qHbPEM2a1pk9aEgvvnckaeLiGVYaHKZccxRaE2E6zGKW26sI5WPcMehCZkiYf4zxfoppWsWDIGAsuptByHT+etkQVEANwNdDM5hIn4Hq0e6gsH1zu68SBhH4FtW4+ukCK3WrHZ0QJBoECU1otWxhP5BqOO8h9OCY2vESxHC2nH+y3vsqVL43JQWmBUv7uaaX4OusdWUZCW8jlYnWd//cmt3RKHh4PCKY7JK0itzoUi+DnnHBN7jxz2FFjX3+BinUeZX3bpXQhw3Xsv1ZXW9aeyGLgzYOuLdBSXmXJcetyJ0heCrOwZUS5AQ0EWWJJSgEIANlJdoChOEMMB7vh3YB0HjrBpK2+iyWC1Y1VCWOpKg/PiYq5GmZDMh1SGq+s/3ZVG5kAYcSKqdHSBqC8sRjY/gU/hysR3w9IU0J5/j1jTFZsw09vTfloEVRAJaAEyOAKvgUw+ fwiGhJsJPS9W8dqieVovqft5DpzNIBztPJxfiDrlHvKbxrwyCEW9+EAHMVJf7F6wqp0rAnlxZQujasL7Urx1Kl0RKT9HT8QHyzTJFpbpuO0hApiWCWD5GCy3AIc0KK3jJIz3ODCVFSGhbLiV0j9FB4j3wmM5JSvsyMhLZAkaFy/kqmlaVeroLWMBbmKUfEA968b37O+kILc3iXYQmMAEQEAAYkBHwQYAQgACQUCWWJJSgIbDAAKCRAtDdo0i3Ae6oyuB/4rh42bgTYncPkMv0l9jsKTB61wwe2Imxfxd52Gk3Ev5slaSFYl2r2FwhTFCX5Ay5XXqoueIG2AlZY2do7jnuLW66dye4GAI67PgKkZfzWJqD+q3bOimnmdlyKM1x2AxGnwPBEdjc4sShFm/uPOPPGJu4f8I9PKjGI62RJUCGHN9KxYJclomhNYe6n1xDsF0e3lDEtWn+KL1WesHO/NY2Mv//qGs9dqIu7rdAXrXiBQaSUz8ol0o4Z924qEWX+23tlWn3XLZPbFPJAB8sPHKWapt3nR/zaPJqfqkqwKWnWFdiKaKIB9GhlXmBlq+0alpjNjbicGlOF1YGF4agbFyoCb X-Rspamd-Queue-Id: 46VK5X33Mtz3Hy0 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=florian-schulze.net header.s=fm1 header.b=Fn7nAUNS; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=hyhHULh9; dmarc=none; spf=pass (mx1.freebsd.org: domain of mail@florian-schulze.net designates 66.111.4.27 as permitted sender) smtp.mailfrom=mail@florian-schulze.net X-Spamd-Result: default: False [-3.58 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[florian-schulze.net:s=fm1,messagingengine.com:s=fm3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_MISSING_CHARSET(2.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.27]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[florian-schulze.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[florian-schulze.net:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-3.48)[ip: (-9.81), ipnet: 66.111.4.0/24(-4.84), asn: 11403(-2.68), country: US(-0.05)]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_LOW(-0.10)[27.4.111.66.list.dnswl.org : 127.0.5.1]; RECEIVED_SPAMHAUS_PBL(0.00)[28.58.245.79.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 15:18:09 -0000 On 6 Sep 2019, at 15:18, Florian Schulze wrote: > Hi! > > On 6 Sep 2019, at 14:43, Konstantin Belousov wrote: > > Thanks for the reply ... > >> On Fri, Sep 06, 2019 at 12:44:46PM +0200, Florian Schulze wrote: >>> Hi! >>> >>> Since FreeBSD 12 (updated from 10.3, I skipped 11.x completely, the = >>> box >>> started around 9.3) I have the issue that ZFS is not freeing up = >>> space >>> for some deleted files. The filesystems where this happens are = >>> mounted >>> into multiple jails via nullfs. Only one jail has write access, the >>> others are read only. When files are deleted the space for them is = >>> not >>> freed. I can still see their objects via zdb. When I unmount one of = >>> the >>> read only nullfs mounts the space is freed and the objects released. >>> >>> I already used lsof, procstat and fstat to see if any process still = >>> has >>> a reference to the file, but that is not the case. But it seems to >>> matter which nullfs mount is unmounted, it is always one of the read >>> only ones. The processes which access the read only mounts are >>> completely different, it only seems to matter that the files are = >>> opened >>> at all. Killing the processes doesn't help, only unmounting the = >>> nullfs. >> There were some bugs in past where nullfs referenced a lower vnode = >> but >> did not dereferenced it. > > Could you be more concrete about "past"? In the past of the 12.0 = > version, or during 11.x? > >>> I have one zfs filesystem where this happens quite often, one were = >>> it >>> happens sometimes and a few others which have a similar setup and = >>> where >>> I never noticed it (though the average file size on them is = >>> smaller). >>> >>> I asked in #freebsd about this and koobs said I should write to this >>> list and CC kib@freebsd.org and mgj@freebsd.org >>> He also did a quick look at the nullfs changes between 10.3 and 12.0 = >>> and >>> spotted the following change, which he said I should mention as = >>> well: >>> https://github.com/freebsd/freebsd/commit/82f9c275c43da09f404546cceef= f187a90ecc573#diff-81e7d6520611101890dd6425324dd8f8 >>> >>> Is there a known bug there? Could the stale NFS handle cause the = >>> leak? >>> Where is that NFS handle coming from? >> >> So what is the exact version of your system ? If 12.0, upgrade = >> kernel >> to latest stable/12 and see if it helps with the leak. > > I'm on 12.0-RELEASE-p3. I will probably be able to try = > 12.0-RELEASE-p10 during the weekend. Not sure I will find the time to = > try and compile my own kernel, or is there a way to do it using = > freebsd-update by using a snapshot? If not, I'd probably rather wait = > for 12.1. It still happens with 12.0-RELEASE-p10. Unless there is a pre-built = kernel to test, I'll wait for 12.1. Regards, Florian Schulze