From owner-freebsd-fs@FreeBSD.ORG Wed May 2 01:58:27 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 553B016A400 for ; Wed, 2 May 2007 01:58:27 +0000 (UTC) (envelope-from staalebk@ifi.uio.no) Received: from smtp.bluecom.no (smtp.bluecom.no [193.75.75.28]) by mx1.freebsd.org (Postfix) with ESMTP id C968713C44C for ; Wed, 2 May 2007 01:58:26 +0000 (UTC) (envelope-from staalebk@ifi.uio.no) Received: from eschew.pusen.org (unknown [193.69.145.10]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.bluecom.no (Postfix) with ESMTP id 70DEC12C0D9 for ; Wed, 2 May 2007 03:58:25 +0200 (CEST) Received: from chiller by eschew.pusen.org with local (Exim 4.50) id 1Hj46u-0001nB-1x for freebsd-fs@freebsd.org; Wed, 02 May 2007 03:58:28 +0200 Date: Wed, 2 May 2007 03:58:27 +0200 From: =?iso-8859-1?Q?St=E5le?= Kristoffersen To: freebsd-fs@freebsd.org Message-ID: <20070502015827.GA5924@eschew.pusen.org> References: <20070427202222.GA26824@eschew.pusen.org> <20070428003115.GA1003@eschew.pusen.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20070428003115.GA1003@eschew.pusen.org> User-Agent: Mutt/1.5.13 (2006-08-11) Subject: Re: ZFS performance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 May 2007 01:58:27 -0000 On 2007-04-28 at 02:31, Ståle Kristoffersen wrote: > cvsup'ed, and buildt a new kernel and now the problem is gone, sorry about > the noise. Not entirely true. I still experience very strange (to me) issues. I'm trying to stream something off the server at about 100 KB/s. It's a video-file located on zfs. I can see that the data pushed across the network is about the correct speed (fluctuating around 100 KB/s). Now if I start up 'zpool iostat -v 1' things start to look strange. it first reads a couple of hundres KB's, then about 1 MB the next second, then 3 MB, then 10 MB then 20 MB, before the cycle restart at around a 3-400 KB again. The numbers differ from run to run but the overall pattern is the same. This makes streaming several streams impossible as it is not able to deliver enough data. (It should easily push 5 streams of 100 KB/s, right?) I'm using samba, but the problem is there when transfering using proftpd, (limiting the transfer on the receiver side to 100 KB/s) alltho the "pattern" is not as clear, it still reads _way_ more data than is transferred. I'm not sure how to debug this, I'm willing to try anything! Below is a snip from 'zpool iostat -v 1', notice that the file beeing streamed is located on the last disc of the pool. (the pool was full, I added one disc and therefor all new data ends up on that disc). It completes two "cycles": capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 0 0 0 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 0 0 0 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 4 0 639K 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 4 0 639K 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 2 0 384K 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 2 0 384K 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 7 0 1023K 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 7 0 1023K 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 27 0 3.50M 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 27 0 3.50M 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 101 0 12.6M 0 ad14 298G 87.4M 0 0 5.99K 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 100 0 12.6M 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 127 0 16.0M 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 127 0 16.0M 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 2 0 384K 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 2 0 384K 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 12 0 1.62M 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 12 0 1.62M 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 27 0 3.50M 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 27 0 3.50M 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 27 0 3.50M 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 27 0 3.50M 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 73 0 9.12M 0 ad14 298G 87.4M 0 0 5.99K 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 72 0 9.12M 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 128 0 16.1M 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 128 0 16.1M 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 5 0 767K 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 5 0 767K 0 ---------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- stash 1.42T 76.3G 7 0 1023K 0 ad14 298G 87.4M 0 0 0 0 ad15 298G 174M 0 0 0 0 ad8 298G 497M 0 0 0 0 ad10s1d 340G 441M 0 0 0 0 ad16 223G 75.1G 7 0 1023K 0 ---------- ----- ----- ----- ----- ----- ----- -- Ståle Kristoffersen staalebk@ifi.uio.no