From owner-freebsd-stable@FreeBSD.ORG Sun Feb 14 19:13:56 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8242B106566B for ; Sun, 14 Feb 2010 19:13:56 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-iw0-f198.google.com (mail-iw0-f198.google.com [209.85.223.198]) by mx1.freebsd.org (Postfix) with ESMTP id 47A8D8FC14 for ; Sun, 14 Feb 2010 19:13:56 +0000 (UTC) Received: by iwn36 with SMTP id 36so2403574iwn.3 for ; Sun, 14 Feb 2010 11:13:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=hqxtx0/knjhPQAZAdYDElrXdstXiUTDadWElrOizKdw=; b=TvAV3eJbFkDeEq6Q2Dzoc3b+KnF8Ri1fz9NzX4H4AVhTZPzxQh1d3VwJKAPn44ciBU FbSkp5kjNYyn4HgBCxRxdJpoQG1LptGvRH5RGArPTnZ9KjTxpSl5foKcVM81OWS3upBD TYkrNByKrra8k+No6bgZ5wFeQMUlpIZs+coW4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=PktzaKjp52EKLApdTPYsoBg52NC3+qgNiVfv2ckfD/DRVJdLEQi7wHtexttXN+HzAJ sBfXAQIK7Zep+JLZerzgReflbjTxWZTKAoKLCozhr7An8UH1QYxGqgyVQqeLVo2Ew8Xf uJHD8hy8wmwQWqlXyq8JzH0cUXNlhknHYjegM= MIME-Version: 1.0 Sender: artemb@gmail.com Received: by 10.231.59.5 with SMTP id j5mr2693222ibh.6.1266174835493; Sun, 14 Feb 2010 11:13:55 -0800 (PST) In-Reply-To: References: Date: Sun, 14 Feb 2010 11:13:55 -0800 X-Google-Sender-Auth: 7aae7f8e9c4863bd Message-ID: From: Artem Belevich To: Jonathan Belson Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: More zfs benchmarks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2010 19:13:56 -0000 Can you check if kstat.zfs.misc.arcstats.memory_throttle_count sysctl increments during your tests? ZFS self-throttles writes if it thinks system is running low on memory. Unfortunately on FreeBSD the 'free' list is a *very* conservative indication of available memory so ZFS often starts throttling before it's really needed. With only 2GB in the system, that's probably what slows you down. The code is in arc_memory_throttle() in sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c, if anyone's curious. --Artem On Sun, Feb 14, 2010 at 9:28 AM, Jonathan Belson wrote= : > Hiya > > After reading some earlier threads about zfs performance, I decided to te= st my own server. =A0I found the results rather surprising... > > The machine is a Dell SC440, dual core 2GHz E2180, 2GB of RAM and ICH7 SA= TA300 controller. =A0There are three Hitachi 500GB drives (HDP725050GLA360)= in a raidz1 configuration (version 13). =A0I'm running amd64 7.2-STABLE fr= om 14th Jan. > > > First of all, I tried creating a 200MB file on / (the only non-zfs partit= ion): > > # dd if=3D/dev/zero of=3D/root/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 6.158355 secs (34053769 bytes/sec) > > # dd if=3D/dev/zero of=3D/root/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 5.423107 secs (38670674 bytes/sec) > > # dd if=3D/dev/zero of=3D/root/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 6.113258 secs (34304982 bytes/sec) > > > Next, I tried creating a 200MB file on a zfs partition: > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 58.540571 secs (3582391 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 46.867240 secs (4474665 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 21.145221 secs (9917853 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 19.387938 secs (10816787 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 21.378161 secs (9809787 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 23.774958 secs (8820844 bytes/sec) > > Ouch! =A0Ignoring the first result, that's still over three times slower = than the non-zfs test. > > > With a 2GB test file: > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 547.901945 secs (3827605 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 595.052017 secs (3524317 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 517.326470 secs (4053827 bytes/sec) > > Even worse :-( > > > Reading 2GB from a raw device: > > dd if=3D/dev/ad4s1a of=3D/dev/null bs=3D1M count=3D2000 > 1024+0 records in > 1024+0 records out > 1073741824 bytes transferred in 13.914145 secs (77169084 bytes/sec) > > > Reading 2GB from a zfs partition (unmounting each time): > > dd if=3D/tank/test/zerofile.000 of=3D/dev/null bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 29.905155 secs (70126772 bytes/sec) > > dd if=3D/tank/test/zerofile.000 of=3D/dev/null bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 32.557361 secs (64414066 bytes/sec) > > dd if=3D/tank/test/zerofile.000 of=3D/dev/null bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 34.137874 secs (61431828 bytes/sec) > > For reading, there seems to be much less of a disparity in performance. > > I notice that one drive is on atapci0 and the other two are on atapci1, b= ut surely it wouldn't make this much of a difference to write speeds? > > Cheers, > > --Jon > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >