Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 18 Mar 2013 18:08:44 +0100
From:      Davide D'Amico <davide.damico@contactlab.com>
To:        kpneal@pobox.com
Cc:        freebsd-fs@freebsd.org
Subject:   Re: FreBSD 9.1 and ZFS v28 performances
Message-ID:  <51474A1C.7090604@contactlab.com>
In-Reply-To: <20130318163833.GA11916@neutralgood.org>
References:  <514729BD.2000608@contactlab.com> <810E5C08C2D149DBAC94E30678234995@multiplay.co.uk> <20130318163833.GA11916@neutralgood.org>

next in thread | previous in thread | raw e-mail | index | archive | help
Il 18/03/13 17:38, kpneal@pobox.com ha scritto:
> On Mon, Mar 18, 2013 at 03:31:51PM -0000, Steven Hartland wrote:
>>
>> ----- Original Message -----
>> From: "Davide D'Amico" <davide.damico@contactlab.com>
>> To: <freebsd-fs@freebsd.org>
>> Sent: Monday, March 18, 2013 2:50 PM
>> Subject: FreBSD 9.1 and ZFS v28 performances
>>
>>
>>> Hi all,
>>> I'm trying to use ZFS on a DELL R720 with 2x6-core, 32GB ram, H710
>>> controller (no JBOD) and 15K rpm SAS HD: I will use it for a mysql 5.6
>>> server, so I am trying to use ZFS to get L2ARC and ZIL benefits.
>>>
>>> I created a RAID10 and used zpool to create a pool on top:
>>>
>>> # zpool create DATA mfid3
>>> # zpool add DATA cache mfid1 log mfid2
>>>
>>> I have a question on zfs performances. Using:
>>>
>>> dd if=/dev/zero of=file.out bs=16k count=1M
>>>
>>> I cannot go faster than 400MB/s so I think I'm missing something; I
>>> tried removing zil, removing l2arc but everything is still the same.
>
> The ZIL only helps with synchronous writes. This is something apps must
> request specifically typically and I would guess that dd would not do that.
> So the ZIL doesn't affect your test.
>
> The L2ARC is a read cache. It does very little for writes. If the ZFS cache
> working set fits all in memory then the L2ARC does nothing for you. Since
> you are writing the only thing needed from the ARC is metadata.
>
>>> mfiutil show volumes:
>>> mfi0 Volumes:
>>>    Id     Size    Level   Stripe  State   Cache   Name
>>>   mfid0 (  278G) RAID-1      64k OPTIMAL Disabled <BASE>
>>>   mfid1 (  118G) RAID-0      64k OPTIMAL Disabled <L2ARC0>
>>>   mfid2 (  118G) RAID-0      64k OPTIMAL Disabled <ZIL0>
>>>   mfid3 ( 1116G) RAID-10   64k OPTIMAL Disabled <DATA>
>>>
>>> zpool status:
>>>    pool: DATA
>>>    state: ONLINE
>>>    scan: none requested
>>> config:
>>>
>>> NAME        STATE     READ WRITE CKSUM
>>> DATA        ONLINE       0     0     0
>>>    mfid3     ONLINE       0     0     0
>>> logs
>>>    mfid2     ONLINE       0     0     0
>>> cache
>>>    mfid1     ONLINE       0     0     0
>
> Warning: your ZIL should probably be mirrored. If it isn't, and the drive
> fails, AND your machine takes a sudden dive (kernel panic, power outage,
> etc) then you will lose data.
I know: here is only one drive because i'm testing, yet.

>
>> DATA  primarycache          metadata               local
>> DATA  secondarycache        all                    default
>
> Is there a specific reason that you are making a point of not putting
> regular data in the ARC? If you do that then reads of data will look in
> the L2ARC, which is a normal 15k drive, before hitting the main pool drives
> which also consists of normal 15k drives. Adding an extra set of spinning
> rust before accessing your spinning rust doesn't sound helpful.
Ok, good point.

>
>> HEAD has some significant changes for the mfi driver specifically:-
>> http://svnweb.freebsd.org/base?view=revision&revision=247369
>>
>> This fixes lots off bugs but also enables full queue support on TBOLT
>> cards so if your mfi is a TBOLT card you may see some speed up in
>> random IO, not that this would effect your test here.
>
> I believe the H710 is a TBOLT card. It was released with the 12G servers
> like the R720.
Anyway I cannot use -CURRENT on a production server, so I should use 9.1.

My 'goal' is to understand if I can perform better than UFS on the same 
hardware setup.

So, I removed the pool and formatted mfid3 using UFS:
# mount | grep mfid3
/dev/mfid3 on /DATA (ufs, local, soft-updates)

where I have:

# dd if=/dev/zero of=file.out bs=8k count=1M
1048576+0 records in
1048576+0 records out
8589934592 bytes transferred in 21.406975 secs (401268025 bytes/sec)

(executed 4-5 times) I have the same throughput than ZFS with or without 
ZIL.

I don't know if this is "normal" or if I am missing something on my 
setup, that's why I'm asking if I can do something more or if with this 
setup this value is the best I can have.

>
> I don't believe the OP mentioned how many drives are in the RAID10. More
> drives ~== more parallelism ~== better performance. So I too am wondering
> how much performance is expected.

>
>> While having a separate ZIL disk is good, your benefits may well be
>> limited if said disk is a traditional HD, better to look at enterprise
>> SSD's for this. The same and them some applies to your L2ARC disks.
>
> Before purchasing SSD's check the H710 docs to make sure they are allowed.
> The 6/i in my R610 specifically says that if an SSD is used it must be the
> only drive. Your R720's H710 is much newer and thus may not have that
> restriction. Still, checking the documentation is cheap.
>

Yes, in R620 SSD are allowed (and a possible choice in the online DELL 
configurator).

Thanks,
d.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51474A1C.7090604>