From owner-freebsd-current@FreeBSD.ORG Wed Nov 11 21:53:03 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A19E81065670 for ; Wed, 11 Nov 2009 21:53:03 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-bw0-f213.google.com (mail-bw0-f213.google.com [209.85.218.213]) by mx1.freebsd.org (Postfix) with ESMTP id 2EC748FC08 for ; Wed, 11 Nov 2009 21:53:02 +0000 (UTC) Received: by bwz5 with SMTP id 5so1610714bwz.3 for ; Wed, 11 Nov 2009 13:53:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:from:date:x-google-sender-auth:message-id:subject:to:cc :content-type:content-transfer-encoding; bh=yKZTz5qDMU80H6nGvepcC119ZZI7a7VErqmM3XN5xE0=; b=w5PHRHAdVbL7qZB8gB5QM3DgRxsBDnX7QssChTTi7FOl8G1Luhj7Pyi3JeNluKWnqX B/PCis9eSlfvktOrg8IUmaN7R8En+3cnJGKTJ/b5k9tN5/t7KyRsX6P59QrHyDSnl5Ut P4/UIGN9nSgaWs5dpWV/Z3jKPlnuC9Pe5m9wI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; b=MckChLtxJhzXF2KwRm4vCmOmqL8hlJSOO2uDX0OnHtBEkMNMmh2afkNvhmUJGZexYo MPlZbpgPVlr/7TQtEUdWeBwKa+VXpkXt6B5gSZmPruQVYAqHh/Vn0nVeDC1FN0mUMw0F E+1RcNb3GNht3tLoPLbUeRobsMlBbyCSxd3pQ= MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.216.90.11 with SMTP id d11mr568547wef.187.1257976381262; Wed, 11 Nov 2009 13:53:01 -0800 (PST) In-Reply-To: <11167f520911111326v13bb442bt36e853afbecdf834@mail.gmail.com> References: <11167f520911111050j36dd94far667c81e6f5c18e69@mail.gmail.com> <20091111204903.GI89052@dan.emsphone.com> <11167f520911111326v13bb442bt36e853afbecdf834@mail.gmail.com> From: Ivan Voras Date: Wed, 11 Nov 2009 22:52:41 +0100 X-Google-Sender-Auth: 9aa8cb309af21964 Message-ID: <9bbcef730911111352t12188bdajbca71bcf35a5beb5@mail.gmail.com> To: "Sam Fourman Jr." Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current@freebsd.org, Dan Nelson Subject: Re: Help ZFS FreeBSD 8.0 RC2 Write performance issue X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2009 21:53:03 -0000 2009/11/11 Sam Fourman Jr. : > On Wed, Nov 11, 2009 at 2:49 PM, Dan Nelson wro= te: >> In the last episode (Nov 11), Ivan Voras said: >>> Sam Fourman Jr. wrote: >>> > I am running FreeBSD 8.0RC2 and I dont understand why my ZFS/NFS is >>> > acting weird on writes. =C2=A0I get ~150mbit writes idk if this is go= od or >>> > not? =C2=A0but it paused for a few seconds every once and awhile. >>> >>> You didn't give any "iostat" statistics - I suspect that if you >>> correlate ifstat and iostat output that you will see that network >>> "pauses" happen during spikes in IO. You should check for this and post >>> your results. >> >> Yes, iostat would be useful here. =C2=A0"iostat -zxC 2" will give you pe= r-disk >> stats plus CPU usage every 2 seconds (CPU may be a factor if you have >> compression enabled). >> >> On a Solaris box I admin, setting zfs_write_limit_override helped stutte= ring >> while doing heavy writes. =C2=A0It's not exported on FreeBSD, but it sho= uld be >> easy to add it as a RW sysctl; it lives in dsl_pool.c and can be tweaked= at >> runtime. =C2=A0Start big and tune it down so each write burst takes unde= r a >> second; it looks like you're writing solid for around 6-8 seconds now. = =C2=A0The >> number will vary depending on your disk speed and how much ARC you have. > > > here are some iostats for you. I do not believe I have compression enable= d > am I mistaken? isn'y SATA2 300MB/s? and I am doing ~6MB/s per disk? > I built this machine with 4GB of memory because I thought ZFS would like = it. > now maybe a re(4) interface isnt the best choice. if that is the > problem here I can change it. Your io/ifstat data is very long but you didn't say if you observed hiccups in network performance during heavy IO times? Since you don't have timestamps in your data you are the only one who can say. > We spent ~$800 on a hardware RAID card thinking that it would help perfor= mance > > Why is it that with sftp we do not see the pauses in Network transfer? I think NFS uses sync disk IO access by default, this may be your problem if you are write-heavy. Try setting vfs.nfsrv.async to 1 to see if this is the cause of your problems.