From nobody Mon Oct 21 13:46:38 2024 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XXGnK1H1Tz5ZJFw for ; Mon, 21 Oct 2024 13:46:45 +0000 (UTC) (envelope-from void@f-m.fm) Received: from fhigh-a8-smtp.messagingengine.com (fhigh-a8-smtp.messagingengine.com [103.168.172.159]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XXGnH6nsCz4Sqq for ; Mon, 21 Oct 2024 13:46:43 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=kGMYOXXH; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=gRz3HXx9; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 103.168.172.159 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from phl-compute-10.internal (phl-compute-10.phl.internal [10.202.2.50]) by mailfhigh.phl.internal (Postfix) with ESMTP id 3CCDA1140277 for ; Mon, 21 Oct 2024 09:46:40 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-10.internal (MEProxy); Mon, 21 Oct 2024 09:46:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :message-id:mime-version:reply-to:subject:subject:to:to; s=fm3; t=1729518400; x=1729604800; bh=Be3lFtQj4Ac4nBA9X27fQn1jkvsDh7xW mmMlno2VRAs=; b=kGMYOXXHt0/eKLKzOHa/qxgvCLTuUHJgw5q3TkTh8wNEbhFZ QPVk0N1BgvSalP9jRwlMwGgULGlIribzL4bo4R3/qIZcgJuTU7wZuneCRuxTsOGB pR+omod7kGc9OVvejmDzXjPgGcqEZ3C7OHQYWgU6DYCW7er2+212G6NJ8qleiS/W PJ7MX3tsBypwieNGys2m7Y2EhI1zAeAa5mCr3Cb0pjaMVguLTZ0L4EUZJKzZqw7R lFyfKw5WPLBwLSmOp96TWsfgL58xuymmgfnVam7u2xPVOUuO8hSZI4GrtMoWr+pF iow8veMdXkTQfllnVqZcg93jLT0anwAF42CxDQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:message-id :mime-version:reply-to:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1729518400; x=1729604800; bh=Be3lFtQj4Ac4nBA9X27fQn1jkvsDh7xWmmM lno2VRAs=; b=gRz3HXx9AU73TvQftRaoDvh4LN2D8dofJijKDyQ/urLgHIqR1+o 0NmjgCXH1WbrS/VKTMIyMker8wV2bhINyb+tQwehWWTKGqtzRAYJZG5ZPL3a+V5S KDMQwEHlfr5W7JcTvV2BNgPHqMQK29a4b+/aAtbaqxp0y2UAg40rprCMrzxmL1cv wA0HiZtz532QW/lazbRlbKSs5jIqeZIIxAL4q2ZUpqsbisDyOfbVdeSleB7jP6CM qMAyLsJHDW94uw0K7UgIX8M9QB4z7oSTcft6wIMSdqPEiGyjJvS/0xbCd7UtPQ9M VMbcVvk/ECiBzvCKCfQ0eQcibW6g1te8/GA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrvdehledgieelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuf fkgggtugesthdtredttddtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhf mheqnecuggftrfgrthhtvghrnhepveduffeivdfffffghfegfeejfefftdeiteehteekfe fhvdefgfettdeuheegffeunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepvhhoihgusehfqdhmrdhfmhdpnhgspghrtghpthhtohepuddpmhhoug gvpehsmhhtphhouhhtpdhrtghpthhtohepfhhrvggvsghsugdqfhhssehfrhgvvggsshgu rdhorhhg X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 21 Oct 2024 09:46:39 -0400 (EDT) Date: Mon, 21 Oct 2024 14:46:38 +0100 From: void To: freebsd-fs@freebsd.org Subject: improving nfs client & server performance Message-ID: Mail-Followup-To: freebsd-fs@freebsd.org List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline X-Spamd-Result: default: False [-3.80 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MID_RHS_NOT_FQDN(0.50)[]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; RWL_MAILSPIKE_VERYGOOD(-0.20)[103.168.172.159:from]; R_SPF_ALLOW(-0.20)[+ip4:103.168.172.128/27]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[103.168.172.159:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_FROM(0.00)[f-m.fm]; RCVD_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; ASN(0.00)[asn:151847, ipnet:103.168.172.0/24, country:AU]; RCVD_VIA_SMTP_AUTH(0.00)[]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim] X-Rspamd-Queue-Id: 4XXGnH6nsCz4Sqq X-Spamd-Bar: --- I'm looking to try to improve nfs client & server performance. The problem I'm seeing is that when clients access the server, if more than one does it at once, the access speed is very 'spiky' and in /var/log/messages on the client, it has things like Oct 20 07:21:55 kernel: nfs server 192.168.1.10:/zroot/share: not responding Oct 20 07:21:55 kernel: nfs server 192.168.1.10:/zroot/share: is alive again Oct 20 07:42:05 kernel: nfs server 192.168.1.10:/zroot/share: not responding Oct 20 07:42:05 kernel: nfs server 192.168.1.10:/zroot/share: is alive again Oct 20 08:29:54 kernel: nfs server 192.168.1.10:/zroot/share: not responding If one client is accessing one file at once, the transfer is very fast. But syncing like rsync or webdav is very problematic and takes much longer than it should. The server is recent 14-stable and exports nfs via the zfs sharenfs property. The clients are a mix of freebsd and linux (debian) I note on the server there's lots of vfs.nfsd sysctl tunables but I'm not sure if thay are relevant in a zfs sharenfs context. There's even more vfs.zfs but nothing pertaining directly to nfs. On a freebsd client, it has these in rc.conf # nfs client stuff nfs_client_enable="YES" Maybe it needs local locks (-L) unsure how to pass flags to the client started in this way. How would I know if local locks were needed? I note in defaults/rc.conf there is nfs_bufpackets="" # bufspace (in packets) for client but I'm not sure what the effects would be. I've so far set 'zfs set sync=disabled' for the particular vdev and 'sysctl vfs.nfsd.maxthreads=256' on the server, about to test this. --