From owner-freebsd-fs@freebsd.org Sat Dec 31 10:26:13 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 822EEC96879 for ; Sat, 31 Dec 2016 10:26:13 +0000 (UTC) (envelope-from jpaetzel@fastmail.net) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5AFA319EA; Sat, 31 Dec 2016 10:26:12 +0000 (UTC) (envelope-from jpaetzel@fastmail.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 18C1E20A69; Sat, 31 Dec 2016 05:26:12 -0500 (EST) Received: from web1 ([10.202.2.211]) by compute4.internal (MEProxy); Sat, 31 Dec 2016 05:26:12 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.net; h=cc :content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= mesmtp; bh=lKcEv4GFdjI57yd6G9D9Vv7H9M0=; b=IL0C6NyUgSO01OUyQw+YS g0FMCJHqelWzRJlmdIJ+Thk9JmR61BNr00CnBHyF8X0VHGBIPqrCrRdAAXQK9zJc cIJFkdM+s/wi3VGu6HB5A9GIg27yXetHYuoJuOleYyeHmvIdGiFhwAXT3qY4yBUb 8iQRj17xWmYkNpf2/p0VBc= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=smtpout; bh=lKcEv4GFdjI57yd6G9D9Vv7H9 M0=; b=W0xw8cpeoNsDG7GebtRbcCITC8caj841ljgYDMIxPzWlT+7PBN5TmWLRu QXoyw8/na6RcqfWrV25iAIBTfn1+3gNcLslOtHvLxbx6L9X/FGkc9X9XxtJUUr+U M9s3tghOshah0FhlQOXPcTReYhv7K6crmDaqY8bV/xdGfXu7NY= X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id EDEC9AA6C5; Sat, 31 Dec 2016 05:26:11 -0500 (EST) Message-Id: <1483179971.3381747.833629401.5EF242B8@webmail.messagingengine.com> From: Josh Paetzel To: freebsd-fs@freebsd.org Cc: mckusick@freebsd.org, Rick Macklem MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-9c115fcf Date: Sat, 31 Dec 2016 04:26:11 -0600 Subject: NFS readdirplus on ZFS with > 1 billion files X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Dec 2016 10:26:13 -0000 We've been chasing this bug for a very long time and finally managed to pin it down. When a ZFS dataset has more than 1 billion files on it and an NFS client does a readdirplus the file handles for files with high znode/inode numbers gets truncated due to a 64 -> 32 bit conversion. https://reviews.freebsd.org/D9009 This isn't a fix so much as a workaround. From a performance standpoint it's the same as if the client mounts with noreaddirplus; sometimes it's a win, sometimes it's a lose. CPU usage does go up on the server a bit. -- Thanks, Josh Paetzel