From owner-freebsd-fs@freebsd.org Sat Apr 11 17:48:41 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1BA2A2BE9EF for ; Sat, 11 Apr 2020 17:48:41 +0000 (UTC) (envelope-from fuz@fuz.su) Received: from fuz.su (fuz.su [IPv6:2001:41d0:8:e508::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "amnesiac", Issuer "amnesiac" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4902Rq4gfvz3Qc8 for ; Sat, 11 Apr 2020 17:48:39 +0000 (UTC) (envelope-from fuz@fuz.su) Received: from fuz.su (localhost [127.0.0.1]) by fuz.su (8.15.2/8.15.2) with ESMTPS id 03BHmVb8054531 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sat, 11 Apr 2020 19:48:31 +0200 (CEST) (envelope-from fuz@fuz.su) Received: (from fuz@localhost) by fuz.su (8.15.2/8.15.2/Submit) id 03BHmVsh054530 for freebsd-fs@freebsd.org; Sat, 11 Apr 2020 19:48:31 +0200 (CEST) (envelope-from fuz) Date: Sat, 11 Apr 2020 19:48:31 +0200 From: Robert Clausecker To: freebsd-fs Subject: Re: ZFS server has gone crazy slow Message-ID: <20200411174831.GA54397@fuz.su> References: <2182C27C-A5D3-41BF-9CE9-7C6883E43074@distal.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <2182C27C-A5D3-41BF-9CE9-7C6883E43074@distal.com> X-Rspamd-Queue-Id: 4902Rq4gfvz3Qc8 X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of fuz@fuz.su has no SPF policy when checking 2001:41d0:8:e508::1) smtp.mailfrom=fuz@fuz.su X-Spamd-Result: default: False [4.27 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[fuz.su]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(0.98)[0.984,0]; IP_SCORE(1.39)[ipnet: 2001:41d0::/32(4.92), asn: 16276(2.03), country: FR(0.00)]; TO_DN_ALL(0.00)[]; NEURAL_SPAM_LONG(1.00)[0.999,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16276, ipnet:2001:41d0::/32, country:FR]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2020 17:48:41 -0000 Hi Chris, What does zpool status say about the pool? Also check if the SMART values of these disks are okay. Yours, Robert Clausecker On Sat, Apr 11, 2020 at 05:36:37PM +0000, Chris Ross wrote: > I have a FreeBSD 11.3-STABLE server that is my router, using a ZFS > mirror (of two GPT disks) as it’s disk. It’s many years old, and has > only been misbehaving like this for a day or so. I’m trying to figure > out what’s wrong. > > I confirmed that internet connectivity isn’t the problem, and a reboot > didn’t fix it. (The reboot took 10-15 minutes to finish going > multi-user, starting daemons, due to the underlying problem described > below.) > > Truss’ing a very basic command (date), I can see that close() and > exit() calls are taking 1-2 seconds. All of the files being opened > are on ZFS, but I don’t know if that’s for sure related. Similarly, > using shell builtin “echo foo” always is immediate, but “/bin/echo” > sometimes works quickly, but sometimes the close() on > /var/run/ld-elf.so.hints takes 3-5 seconds. > > I _think_ this is a filesystem problem. It’s very hard to diagnose > because logging in, and doing anything, takes many seconds per > command. zpool status shows my mirror as online, so I’m not sure > where I should check. > > I’d appreciate any help! Thanks much… > > - Chris > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > -- () ascii ribbon campaign - for an 8-bit clean world /\ - against html email - against proprietary attachments