From owner-freebsd-stable@freebsd.org Thu Nov 26 09:19:30 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9DD4A39379 for ; Thu, 26 Nov 2015 09:19:30 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (unknown [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47F251EB3 for ; Thu, 26 Nov 2015 09:19:29 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from bsdrookie.norma.com. ([IPv6:fd00::77d]) by elf.hq.norma.perm.ru (8.14.9/8.14.9) with ESMTP id tAQ9JIOH018220 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 26 Nov 2015 14:19:19 +0500 (YEKT) (envelope-from emz@norma.perm.ru) To: freebsd-stable From: "Eugene M. Zheganin" Subject: high disk %busy, while almost nothing happens X-Enigmail-Draft-Status: N1110 Message-ID: <5656CE96.9000103@norma.perm.ru> Date: Thu, 26 Nov 2015 14:19:18 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (elf.hq.norma.perm.ru [IPv6:fd00::30a]); Thu, 26 Nov 2015 14:19:19 +0500 (YEKT) X-Spam-Status: No hits=-100.4 bayes=0.0000 testhits AWL=0.020,BAYES_00=-1.9, RDNS_NONE=0.793,SPF_SOFTFAIL=0.665,USER_IN_WHITELIST=-100 autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on elf.hq.norma.perm.ru X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Nov 2015 09:19:31 -0000 Hi. I'm using FreeBSD 10.1-STABLE as an application server, last week I've noticed that disks are always busy while gstat shows that the activity measured in iops/reads/writes is low, form my point of view: L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 8 56 50 520 160.6 6 286 157.4 100.2 gpt/zfsroot0 8 56 51 1474 162.8 5 228 174.4 99.9 gpt/zfsroot1 These %busy numbers arent't changing much, and from my point of view both disks do very little. zpool iostat: [root@gw0:~]# zpool iostat 1 capacity operations bandwidth pool alloc free read write read write ---------- ----- ----- ----- ----- ----- ----- zfsroot 270G 186G 90 131 1,17M 1,38M ---------- ----- ----- ----- ----- ----- ----- zfsroot 270G 186G 113 93 988K 418K ---------- ----- ----- ----- ----- ----- ----- zfsroot 270G 186G 112 0 795K 93,8K ---------- ----- ----- ----- ----- ----- ----- zfsroot 270G 186G 109 55 1,28M 226K ---------- ----- ----- ----- ----- ----- ----- zfsroot 270G 186G 112 116 1,36M 852K ---------- ----- ----- ----- ----- ----- ----- zfsroot 270G 186G 105 47 1,44M 1,61M ---------- ----- ----- ----- ----- ----- ----- What can cause this ? Pool is fragmented indeed, but I have others server with comparable amount of fragmentation, and no signs of busyness while reads/writes are that low. # zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zfsroot 456G 270G 186G - 51% 59% 1.00x ONLINE - Loader settings: vfs.root.mountfrom="zfs:zfsroot" vfs.zfs.arc_max="2048M" vfs.zfs.zio.use_uma=1 I've tried to play with vfs.zfs.zio.use_uma, but without any noticeable effect. I've also tried to add separate log devices - this didn't help either. Thanks. Eugene.