From nobody Sun Oct 16 22:33:28 2022 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MrFKC2rYCz4gFj5 for ; Sun, 16 Oct 2022 22:33:51 +0000 (UTC) (envelope-from void@f-m.fm) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4MrFKB2tRNz3Zq6 for ; Sun, 16 Oct 2022 22:33:50 +0000 (UTC) (envelope-from void@f-m.fm) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 331CB3200708 for ; Sun, 16 Oct 2022 18:33:49 -0400 (EDT) Received: from imap46 ([10.202.2.96]) by compute3.internal (MEProxy); Sun, 16 Oct 2022 18:33:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm3; t=1665959628; x=1666046028; bh=08wsmMCsEk ZgfX1BRHVquc+dGhjWUMRs2AulpxxIyNA=; b=E2Ay+iZM1fFs6Avw1D40dytdsG hcloDczyV4FtvNNMsv6fX5/NF2X/0V4NrGQBm0ZHow76tGXraaaOvMJ0+aLAJ/0i wagqvpDtwdS3cu89Hosk0jhhko9yq/K0D6+uaqbbbYMu2hRIXK7uU42A2XE4QWl7 ZIrPCFKAFFCDt3WQr+K2/JRBUU6aaCQTKtb6gRkCeDy1UdkEmThDdOV1LkhORq+s EqiFCaFqb0Zv1hzAXWP4nXaWyeghfjjHTOZsRFlWyCmNvYf+aSOOI92NPEeXSRY7 LPQiWWrk4Yhg+6pXEyVnZJlrZiwvh4SshLhkXRrMR7uDgshYkho2xp4YbxnA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1665959628; x=1666046028; bh=08wsmMCsEkZgfX1BRHVquc+dGhjW UMRs2AulpxxIyNA=; b=b2D04GH9FBxvb0Gi8SFoVkNDUKmuVoHLW9buaq+c1ku9 /lZ5VYJwSlnJS6XqBdyC6xmyiIni9sBzf8l3tfi5Brb+nIETWIp4W6hJq8e5m4qS Hw3jGEirQptFqIh/k/RPgwz+AhpBlYpSZyTCFIhSGMlSkSL2pciEP8HuiMr47teW Dm1hGE+OoViu7xIbvHyaMqyuyc/1TqyYdzLxgfLSwDo4fSiATNHng1MWcIjqQxBZ s01nwRDCBNjskSwagu4+Pw1AP/ZXrHg9cP+uyDK21uPN5QXmSjyDnltVdtjdK+X/ UFXxDHgZJOU7d9x1xKAEBnhwK90GbaXR6YrAao5tGg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfeekkedguddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffr rghtthgvrhhnpeeitedvueehtdehtddvhfeuhfevhedvieelvdeiffehveelheegfedule ejudekvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhm pehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 87A582A20079; Sun, 16 Oct 2022 18:33:48 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.7.0-alpha0-1047-g9e4af4ada4-fm-20221005.001-g9e4af4ad List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 Message-Id: <6e408755-7d8b-496a-a34f-b30dd4ea9b53@app.fastmail.com> In-Reply-To: References: <1625dc7b-81c4-4350-8f86-1b65f5a860d9@app.fastmail.com> <64a0af42-630d-4f42-8b8c-921b68b03078@app.fastmail.com> <528a0bcc-6bcd-4d66-aeff-273542deb21d@app.fastmail.com> Date: Sun, 16 Oct 2022 22:33:28 +0000 From: void To: freebsd-fs Subject: Re: zfs with operations like rm -rf takes a very long time recently Content-Type: text/plain X-Rspamd-Queue-Id: 4MrFKB2tRNz3Zq6 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=E2Ay+iZM; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=b2D04GH9; dmarc=pass (policy=none) header.from=f-m.fm; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 64.147.123.19 as permitted sender) smtp.mailfrom=void@f-m.fm X-Spamd-Result: default: False [-4.59 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.995]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19:c]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.19:from]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; ARC_NA(0.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_COUNT_THREE(0.00)[4]; FREEMAIL_FROM(0.00)[f-m.fm]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org] X-ThisMailContainsUnwantedMimeParts: N On Sun, 16 Oct 2022, at 22:06, Warner Losh wrote: > The disk is quite busy and performing decently well for the load... > The quite busy > part I see others are trying to explain and/or diagnose, so I'll leave > them to it. It runs ssl/nextcloud/postgresql13/nginx/memcached/exim but for only a few users. I'd not say it's particularly busy usually. last pid: 5664; load averages: 0.31, 0.24, 0.20 up 0+05:29:57 23:30:10 69 processes: 1 running, 68 sleeping CPU: 0.7% user, 0.0% nice, 1.9% system, 0.1% interrupt, 97.4% idle Mem: 391M Active, 237M Inact, 5304M Wired, 40K Buf, 1882M Free ARC: 2623M Total, 1857M MFU, 149M MRU, 143K Anon, 79M Header, 537M Other 1587M Compressed, 21G Uncompressed, 13.30:1 Ratio Swap: 16G Total, 16G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 4369 www 1 23 0 480M 148M accept 0 0:45 2.20% php-fpm 1451 www 1 68 0 514M 151M accept 2 0:37 0.00% php-fpm 5016 www 1 34 0 478M 145M accept 1 0:20 0.00% php-fpm 1130 unbound 1 20 0 153M 119M kqread 0 0:10 0.00% unbound 1292 root 3 20 0 19M 3320K select 1 0:09 0.00% apcupsd 1345 postgres 1 23 0 2153M 74M select 3 0:05 0.00% postgres 1351 postgres 1 20 0 31M 10M kqread 3 0:03 0.00% postgres 1474 nobody 10 20 0 51M 7188K kqread 3 0:03 0.00% memcached