Date: Fri, 11 Jan 2013 22:43:46 +0800 (SGT) From: Patrick Dung <patrick_dkt@yahoo.com.hk> To: Tom Evans <tevans.uk@googlemail.com> Cc: freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: ZFS sub-optimal performance with default setting Message-ID: <1357915426.16602.YahooMailClassic@web190801.mail.sg3.yahoo.com>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
Hi Tom,
To make it simple, I have setup in this setting:
Host: Intel dual core 3Ghz CPU, RHEL 6.3 x64, RAM 8GB
Freebsd 9.1 -i386 VM with these setting:
CPU: One
Memory: 2GB
5GB for OS (da1)
5GB for ZFS (da2), no separate ZIL
Installed software:
Postgresql 9.2.2 (compile from ports) /usr/local/pgsql is a ZFS dataset
OTRS 3.1.6 (compile from ports)
Apache 2 install from packages
zfs/postgresql/otrs/apache is in default setting, except I have turned off atime in ZFS.
I have run OTRS benchmark twice, below is the result:
Insert Time: 10000 12 s :-( Should not take more than 5's on an average system.
Update Time: 10000 7 s Ok
Select Time: 10000 3 s :-) Looks fine!
Delete Time: 10000 2 s :-) Looks fine!
Thanks,
Patrick
--- On Fri, 1/11/13, Tom Evans <tevans.uk@googlemail.com> wrote:
From: Tom Evans <tevans.uk@googlemail.com>
Subject: Re: ZFS sub-optimal performance with default setting
To: "Patrick Dung" <patrick_dkt@yahoo.com.hk>
Cc: "freebsd-fs" <freebsd-fs@freebsd.org>
Date: Friday, January 11, 2013, 1:08 AM
On Thu, Jan 10, 2013 at 4:54 PM, Patrick Dung <patrick_dkt@yahoo.com.hk> wrote:
> I have tried some tests, good and bad result is in below..........
> I am sure there is some bottleneck, and the root cause is still unknown.
>
Hi Patrick
Correct me if I've made a mistake, but have you shown how you have
configured your ZFS setup? Number and type of disks, etc, mirrored,
raidz or raidz2? The output of zpool status and zfs-stats (from ports)
would be useful.
Cheers
Tom
[-- Attachment #2 --]
# zfs get all
NAME PROPERTY VALUE SOURCE
data type filesystem -
data creation Fri Jan 11 21:25 2013 -
data used 175M -
data available 2.77G -
data referenced 108M -
data compressratio 1.00x -
data mounted yes -
data quota none default
data reservation none default
data recordsize 128K default
data mountpoint /data default
data sharenfs off default
data checksum on default
data compression off default
data atime off local
data devices on default
data exec on default
data setuid on default
data readonly off default
data jailed off default
data snapdir hidden default
data aclmode discard default
data aclinherit restricted default
data canmount on default
data xattr off temporary
data copies 1 default
data version 5 -
data utf8only off -
data normalization none -
data casesensitivity sensitive -
data vscan off default
data nbmand off default
data sharesmb off default
data refquota none default
data refreservation none default
data primarycache all default
data secondarycache all default
data usedbysnapshots 0 -
data usedbydataset 108M -
data usedbychildren 66.7M -
data usedbyrefreservation 0 -
data logbias latency default
data dedup off default
data mlslabel -
data sync standard default
data refcompressratio 1.00x -
data written 108M -
data/pgsql type filesystem -
data/pgsql creation Fri Jan 11 21:27 2013 -
data/pgsql used 66.5M -
data/pgsql available 2.77G -
data/pgsql referenced 66.5M -
data/pgsql compressratio 1.00x -
data/pgsql mounted yes -
data/pgsql quota none default
data/pgsql reservation none default
data/pgsql recordsize 128K default
data/pgsql mountpoint /usr/local/pgsql local
data/pgsql sharenfs off default
data/pgsql checksum on default
data/pgsql compression off default
data/pgsql atime off inherited from data
data/pgsql devices on default
data/pgsql exec on default
data/pgsql setuid on default
data/pgsql readonly off default
data/pgsql jailed off default
data/pgsql snapdir hidden default
data/pgsql aclmode discard default
data/pgsql aclinherit restricted default
data/pgsql canmount on default
data/pgsql xattr off temporary
data/pgsql copies 1 default
data/pgsql version 5 -
data/pgsql utf8only off -
data/pgsql normalization none -
data/pgsql casesensitivity sensitive -
data/pgsql vscan off default
data/pgsql nbmand off default
data/pgsql sharesmb off default
data/pgsql refquota none default
data/pgsql refreservation none default
data/pgsql primarycache all default
data/pgsql secondarycache all default
data/pgsql usedbysnapshots 0 -
data/pgsql usedbydataset 66.5M -
data/pgsql usedbychildren 0 -
data/pgsql usedbyrefreservation 0 -
data/pgsql logbias latency default
data/pgsql dedup off default
data/pgsql mlslabel -
data/pgsql sync standard default
data/pgsql refcompressratio 1.00x -
data/pgsql written 66.5M -
[-- Attachment #3 --]
# zpool status
pool: data
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
da1 ONLINE 0 0 0
errors: No known data errors
[-- Attachment #4 --]
# zfs-stats -a
------------------------------------------------------------------------
ZFS Subsystem Report Fri Jan 11 22:38:47 2013
------------------------------------------------------------------------
System Information:
Kernel Version: 901000 (osreldate)
Hardware Platform: i386
Processor Architecture: i386
ZFS Storage pool Version: 28
ZFS Filesystem Version: 5
FreeBSD 9.1-RELEASE #0 r243826: Tue Dec 4 06:55:39 UTC 2012 root
10:38PM up 9 mins, 1 user, load averages: 0.00, 0.10, 0.10
------------------------------------------------------------------------
System Memory:
1.37% 27.39 MiB Active, 1.06% 21.18 MiB Inact
5.19% 103.51 MiB Wired, 0.00% 0 Cache
92.36% 1.80 GiB Free, 0.02% 372.00 KiB Gap
Real Installed: 2.00 GiB
Real Available: 99.19% 1.98 GiB
Real Managed: 98.21% 1.95 GiB
Logical Total: 2.00 GiB
Logical Used: 9.00% 184.25 MiB
Logical Free: 91.00% 1.82 GiB
Kernel Memory: 82.84 MiB
Data: 79.12% 65.54 MiB
Text: 20.88% 17.29 MiB
Kernel Memory Map: 411.76 MiB
Size: 18.52% 76.25 MiB
Free: 81.48% 335.50 MiB
------------------------------------------------------------------------
ARC Summary: (HEALTHY)
Memory Throttle Count: 0
ARC Misc:
Deleted: 6
Recycle Misses: 0
Mutex Misses: 0
Evict Skips: 0
ARC Size: 8.18% 21.07 MiB
Target Size: (Adaptive) 100.00% 257.50 MiB
Min Size (Hard Limit): 12.50% 32.19 MiB
Max Size (High Water): 8:1 257.50 MiB
ARC Size Breakdown:
Recently Used Cache Size: 50.00% 128.75 MiB
Frequently Used Cache Size: 50.00% 128.75 MiB
ARC Hash Breakdown:
Elements Max: 580
Elements Current: 100.00% 580
Collisions: 24
Chain Max: 1
Chains: 3
------------------------------------------------------------------------
ARC Efficiency: 5.46k
Cache Hit Ratio: 92.94% 5.07k
Cache Miss Ratio: 7.06% 385
Actual Hit Ratio: 92.87% 5.07k
Data Demand Efficiency: 85.86% 1.61k
Data Prefetch Efficiency: 0.00% 3
CACHE HITS BY CACHE LIST:
Anonymously Used: 0.08% 4
Most Recently Used: 42.79% 2.17k
Most Frequently Used: 57.13% 2.90k
Most Recently Used Ghost: 0.00% 0
Most Frequently Used Ghost: 0.00% 0
CACHE HITS BY DATA TYPE:
Demand Data: 27.29% 1.38k
Prefetch Data: 0.00% 0
Demand Metadata: 72.63% 3.68k
Prefetch Metadata: 0.08% 4
CACHE MISSES BY DATA TYPE:
Demand Data: 59.22% 228
Prefetch Data: 0.78% 3
Demand Metadata: 31.95% 123
Prefetch Metadata: 8.05% 31
------------------------------------------------------------------------
L2ARC is disabled
------------------------------------------------------------------------
------------------------------------------------------------------------
VDEV cache is disabled
------------------------------------------------------------------------
ZFS Tunables (sysctl):
kern.maxusers 384
vm.kmem_size 432013312
vm.kmem_size_scale 3
vm.kmem_size_min 0
vm.kmem_size_max 432013312
vfs.zfs.l2c_only_size 0
vfs.zfs.mfu_ghost_data_lsize 1100288
vfs.zfs.mfu_ghost_metadata_lsize 151552
vfs.zfs.mfu_ghost_size 1251840
vfs.zfs.mfu_data_lsize 2483200
vfs.zfs.mfu_metadata_lsize 113664
vfs.zfs.mfu_size 2596864
vfs.zfs.mru_ghost_data_lsize 131072
vfs.zfs.mru_ghost_metadata_lsize 372736
vfs.zfs.mru_ghost_size 503808
vfs.zfs.mru_data_lsize 16656896
vfs.zfs.mru_metadata_lsize 773632
vfs.zfs.mru_size 18518016
vfs.zfs.anon_data_lsize 0
vfs.zfs.anon_metadata_lsize 0
vfs.zfs.anon_size 16384
vfs.zfs.l2arc_norw 1
vfs.zfs.l2arc_feed_again 1
vfs.zfs.l2arc_noprefetch 1
vfs.zfs.l2arc_feed_min_ms 200
vfs.zfs.l2arc_feed_secs 1
vfs.zfs.l2arc_headroom 2
vfs.zfs.l2arc_write_boost 8388608
vfs.zfs.l2arc_write_max 8388608
vfs.zfs.arc_meta_limit 67502080
vfs.zfs.arc_meta_used 2956892
vfs.zfs.arc_min 33751040
vfs.zfs.arc_max 270008320
vfs.zfs.dedup.prefetch 1
vfs.zfs.mdcomp_disable 0
vfs.zfs.write_limit_override 0
vfs.zfs.write_limit_inflated 6390251520
vfs.zfs.write_limit_max 266260480
vfs.zfs.write_limit_min 33554432
vfs.zfs.write_limit_shift 3
vfs.zfs.no_write_throttle 0
vfs.zfs.zfetch.array_rd_sz 1048576
vfs.zfs.zfetch.block_cap 256
vfs.zfs.zfetch.min_sec_reap 2
vfs.zfs.zfetch.max_streams 8
vfs.zfs.prefetch_disable 1
vfs.zfs.mg_alloc_failures 8
vfs.zfs.check_hostid 1
vfs.zfs.recover 0
vfs.zfs.txg.synctime_ms 1000
vfs.zfs.txg.timeout 5
vfs.zfs.vdev.cache.bshift 16
vfs.zfs.vdev.cache.size 0
vfs.zfs.vdev.cache.max 16384
vfs.zfs.vdev.write_gap_limit 4096
vfs.zfs.vdev.read_gap_limit 32768
vfs.zfs.vdev.aggregation_limit 131072
vfs.zfs.vdev.ramp_rate 2
vfs.zfs.vdev.time_shift 6
vfs.zfs.vdev.min_pending 4
vfs.zfs.vdev.max_pending 10
vfs.zfs.vdev.bio_flush_disable 0
vfs.zfs.cache_flush_disable 0
vfs.zfs.zil_replay_disable 0
vfs.zfs.zio.use_uma 0
vfs.zfs.snapshot_list_prefetch 0
vfs.zfs.version.zpl 5
vfs.zfs.version.spa 28
vfs.zfs.version.acl 1
vfs.zfs.debug 0
vfs.zfs.super_owner 0
------------------------------------------------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1357915426.16602.YahooMailClassic>