From owner-freebsd-fs@FreeBSD.ORG Wed Nov 17 15:14:58 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78D081065679 for ; Wed, 17 Nov 2010 15:14:58 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 04C5F8FC14 for ; Wed, 17 Nov 2010 15:14:57 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PIjiq-000638-Lp for freebsd-fs@freebsd.org; Wed, 17 Nov 2010 16:14:56 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 17 Nov 2010 16:14:56 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 17 Nov 2010 16:14:56 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Wed, 17 Nov 2010 16:14:50 +0100 Lines: 45 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6 X-Enigmail-Version: 1.1.2 Subject: Kind of slow IO under pressure? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Nov 2010 15:14:58 -0000 I'm running bonnie++ on UFS hosted on a FC SAN, and during the "rewrite" phase of the benchmark (bonnie++ is deliberately trying to trash disk caches), the situation from "top" is like this: 21 root 76 - 0K 16K qsleep 1 2:48 44.38% bufdaemon 1274 root 76 0 11296K 1716K getblk 7 3:35 41.06% bonnie++ 3 root -8 - 0K 16K - 2 1:44 21.78% g_up With disk rates of about 40 MB/s in either direction and around 300 IOPS. Together these three processes take up a whole CPU core and bonnie++ is single-threaded, which sort of suggests that this supposedly IO benchmark is actually CPU-bound on this 8-core 2.4 GHz system. Transaction sizes are curiously not-quite-64KiB: tty da0 da1 da2 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 380 15.93 28 0.44 63.75 1242 77.30 1.93 0 0.00 0 0 9 0 91 0 1510 16.00 3 0.05 63.86 1148 71.56 0.00 0 0.00 0 0 10 0 90 0 1063 16.00 2 0.03 63.96 1093 68.24 0.00 0 0.00 0 0 14 0 86 0 357 0.00 0 0.00 63.95 1024 63.95 0.00 0 0.00 0 0 15 0 85 Interestingly, binding (with cpuset) all three processes to the same CPU makes things very strange, with bufdaemon climbing to 90% CPU usage and bandwidth dropping to ~~ 5 MB/s (but not completely stalling). Relaxing bufdeamon binds to two CPUs makes things normal again. On the "read" phase the situation is: 1274 root 61 0 11296K 1704K getblk 0 7:05 26.56% bonnie++ 18 root 44 - 0K 16K psleep 4 0:09 2.88% pagedaemon 3 root -8 - 0K 16K - 0 3:31 1.66% g_up 4 root -8 - 0K 16K - 2 0:17 1.66% g_down 12 root -40 - 0K 432K WAIT 5 0:11 1.27% {swi2: cambio} i.e. no CPU hogging and the performance is ~~ 140 MB/s. The write phase is similar, nothing unusual.