From owner-freebsd-fs@freebsd.org Fri Aug 11 06:28:30 2017
Return-Path: <owner-freebsd-fs@freebsd.org>
Delivered-To: freebsd-fs@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
[IPv6:2001:1900:2254:206a::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 607A9DC563C;
Fri, 11 Aug 2017 06:28:30 +0000 (UTC)
(envelope-from emz@norma.perm.ru)
Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256
bits)) (Client CN "mail.norma.perm.ru",
Issuer "Vivat-Trade UNIX Root CA" (not verified))
by mx1.freebsd.org (Postfix) with ESMTPS id BB5953538;
Fri, 11 Aug 2017 06:28:29 +0000 (UTC)
(envelope-from emz@norma.perm.ru)
Received: from bsdrookie.norma.com. (net206-94.perm.ertelecom.ru
[46.146.206.94] (may be forged))
by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPS id v7B6SMEH008220
(version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO);
Fri, 11 Aug 2017 11:28:23 +0500 (YEKT)
(envelope-from emz@norma.perm.ru)
To: freebsd-fs@freebsd.org
Cc: freebsd-stable <freebsd-stable@FreeBSD.org>
From: "Eugene M. Zheganin" <emz@norma.perm.ru>
Subject: zfs listing and CPU
Message-ID: <aa26e888-05ef-c876-abf3-778ff08f4857@norma.perm.ru>
Date: Fri, 11 Aug 2017 11:28:21 +0500
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101
Thunderbird/52.1.1
MIME-Version: 1.0
Content-Type: text/plain; charset=koi8-r; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-Spamd-Result: default: False [1.50 / 25.00]
RBL_SPAMHAUS_PBL(2.00)[94.206.146.46.zen.spamhaus.org : 127.0.0.10]
HFILTER_HOSTNAME_UNKNOWN(2.50)[] BAYES_HAM(-3.00)[99.99%]
DMARC_NA(0.00)[norma.perm.ru] MIME_GOOD(-0.10)[text/plain]
R_DKIM_NA(0.00)[] R_SPF_SOFTFAIL(0.00)[~all]
MID_RHS_MATCH_FROM(0.00)[]
RECEIVED_SPAMHAUS(0.00)[94.206.146.46.zen.spamhaus.org]
TO_DN_SOME(0.00)[] RCPT_COUNT_2(0.00)[]
TO_MATCH_ENVRCPT_ALL(0.00)[] FROM_HAS_DN(0.00)[]
FROM_EQ_ENVFROM(0.00)[] RCVD_COUNT_1(0.00)[]
ONCE_RECEIVED(0.10)[]
X-Rspamd-Server: localhost
X-Rspamd-Scan-Time: 0.93
X-Rspamd-Queue-ID: v7B6SMEH008220
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-fs>,
<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs/>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 11 Aug 2017 06:28:30 -0000
Hi,
Why does the zfs listing eat so much of the CPU ?
last pid: 47151; load averages: 3.97, 6.35,
6.13 up 1+23:21:18 09:15:13
146 processes: 3 running, 142 sleeping, 1 waiting
CPU: 0.0% user, 0.0% nice, 30.5% system, 0.3% interrupt, 69.2% idle
Mem: 44M Active, 360M Inact, 37G Wired, 25G Free
ARC: 32G Total, 14G MFU, 17G MRU, 312M Anon, 803M Header, 523M Other
Swap: 32G Total, 185M Used, 32G Free
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
11 root 32 155 ki31 0K 512K CPU0 0 1104.1 2666.23%
idle
0 root 9880 -16 - 0K 154M swapin 11 314.0H 281.29%
kernel
13 root 3 -8 - 0K 48K gread 0 20.9H 28.71% geom
47114 root 1 20 0 40432K 3840K db->db 4 0:05 26.84% zfs
47099 root 1 20 0 40432K 3840K zio->i 17 0:05 26.83% zfs
47106 root 1 20 0 40432K 3840K db->db 21 0:05 26.81% zfs
47150 root 1 20 0 40432K 3428K db->db 13 0:03 26.31% zfs
47141 root 1 20 0 40432K 3428K zio->i 28 0:03 26.31% zfs
47135 root 1 20 0 40432K 3312K g_wait 9 0:03 25.51% zfs
4 root 7 -16 - 0K 112K - 20 975:01 19.73% cam
5 root 2494 -8 - 0K 39952K arc_re 18 20.2H 18.58%
zfskern
12 root 65 -60 - 0K 1040K WAIT 0 17.8H 15.64% intr
22 root 2 -16 - 0K 32K psleep 3 66:34 7.31%
pagedaemon
590 root 10 -16 - 0K 160K - 21 177:02 2.96% ctl
[...]
This is from winter 2017 11-STABLE (r310734), one of the 'zfs'es is
cloning, and all the others are 'zfs list -t all'. I have like 25 gigs
of free RAM, do I have any chance of speeding this up using may be some
caching or some sysctl tuning ? We are using a simple ZFS web API that
may issue concurrent or sequential listing requests, so as you can see
they sometimes do stack.
Thanks.
Eugene.