From owner-freebsd-fs@freebsd.org Sun Dec 23 10:30:59 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E34D135583A for ; Sun, 23 Dec 2018 10:30:59 +0000 (UTC) (envelope-from alexader@leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:313::1:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0D1426D1AC for ; Sun, 23 Dec 2018 10:30:55 +0000 (UTC) (envelope-from alexader@leidinger.net) Received: from localhost (IO.Leidinger.net [192.168.1.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by outgoing.leidinger.net (Postfix) with ESMTPS id 65CEBCD80; Sun, 23 Dec 2018 11:30:33 +0100 (CET) Date: Sun, 23 Dec 2018 11:30:31 +0100 From: Alexander Leidinger To: Rick Macklem Cc: Peter Eriksson , "freebsd-fs@freebsd.org" Subject: Re: Suggestion for hardware for ZFS fileserver Message-ID: <20181223113031.00005150@Leidinger.net> In-Reply-To: References: <4f816be7-79e0-cacb-9502-5fbbe343cfc9@denninger.net> <3160F105-85C1-4CB4-AAD5-D16CF5D6143D@ifm.liu.se> X-Mailer: Claws Mail 3.17.1 (GTK+ 2.24.32; x86_64-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Leidinger-net-MailScanner-Information: Please contact the ISP for more information X-Leidinger-net-MailScanner-ID: CAB3315507.A50D9 X-Leidinger-net-MailScanner: Found to be clean X-Leidinger-net-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-3, required 6, autolearn=not spam, ALL_TRUSTED -1.00, BAYES_00 -1.90, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10) X-Leidinger-net-MailScanner-From: alexader@leidinger.net X-Leidinger-net-MailScanner-Watermark: 1546165840.18832@pOB+g+IpQ6LyTfkIjRt1gQ X-Leidinger-net-Spam-Status: No X-Rspamd-Queue-Id: 0D1426D1AC X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.91 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[leidinger.net:s=outgoing-alex]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.78)[0.784,0]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[leidinger.net:+]; DMARC_POLICY_ALLOW(-0.50)[Leidinger.net,quarantine]; MX_GOOD(-0.01)[mailgate.leidinger.net]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-0.69)[asn: 34240(-3.43), country: DE(-0.01)]; ASN(0.00)[asn:34240, ipnet:2a00:1828::/32, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Dec 2018 10:30:59 -0000 On Fri, 21 Dec 2018 23:49:58 +0000 Rick Macklem wrote: > Peter Eriksson wrote: > [good stuff snipped] > >This has caused some interesting problems… > > > >First thing we noticed was that booting would take forever… Mounting > >the 20-100k >filesystems _and_ enabling them to be shared via NFS is > >not done efficient at all (for >each filesystem it > >re-reads /etc/zfs/exports (a couple of times) befor appending one > >>line to the end. Repeat 20-100,000 times… Not to mention the big > >>kernel lock for >NFS “hold all NFS activity while we flush and > >>reinstalls all sharing information per >filesystem” being done by > >>mountd… > Yes, /etc/exports and mountd were implemented in the 1980s, when a > dozen file systems would have been a large server. Scaling to 10,000 > or more file systems wasn't even conceivable back then. There was an effort in 2012 to improve on that. https://sourceforge.net/projects/nfse/ I can't find code for this. In GNATS this was as kern/136865, but I can't find it in bugzilla. It was not picked up by us, no idea why (maybe nobody bothered back then). Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF