Date: Sun, 7 Sep 2014 20:01:49 +0400 From: Slawa Olhovchenkov <slw@zxy.spb.ru> To: Steven Hartland <smh@FreeBSD.org> Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r256956 - in head/sys: cam/ata cam/scsi cddl/contrib/opensolaris/uts/common/fs/zfs cddl/contrib/opensolaris/uts/common/fs/zfs/sys geom sys Message-ID: <20140907160149.GC72244@zxy.spb.ru> In-Reply-To: <201310230954.r9N9sxAZ011750@svn.freebsd.org> References: <201310230954.r9N9sxAZ011750@svn.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Oct 23, 2013 at 09:54:59AM +0000, Steven Hartland wrote: > Author: smh > Date: Wed Oct 23 09:54:58 2013 > New Revision: 256956 > URL: http://svnweb.freebsd.org/changeset/base/256956 > > Log: > Improve ZFS N-way mirror read performance by using load and locality > information. > > The existing algorithm selects a preferred leaf vdev based on offset of the zio > request modulo the number of members in the mirror. It assumes the devices are > of equal performance and that spreading the requests randomly over both drives > will be sufficient to saturate them. In practice this results in the leaf vdevs > being under utilized. > > The new algorithm takes into the following additional factors: > * Load of the vdevs (number outstanding I/O requests) > * The locality of last queued I/O vs the new I/O request. > > Within the locality calculation additional knowledge about the underlying vdev > is considered such as; is the device backing the vdev a rotating media device. > > This results in performance increases across the board as well as significant > increases for predominantly streaming loads and for configurations which don't > have evenly performing devices. > > The following are results from a setup with 3 Way Mirror with 2 x HD's and > 1 x SSD from a basic test running multiple parrallel dd's. > > With pre-fetch disabled (vfs.zfs.prefetch_disable=1): > > == Stripe Balanced (default) == > Read 15360MB using bs: 1048576, readers: 3, took 161 seconds @ 95 MB/s > == Load Balanced (zfslinux) == > Read 15360MB using bs: 1048576, readers: 3, took 297 seconds @ 51 MB/s > == Load Balanced (locality freebsd) == > Read 15360MB using bs: 1048576, readers: 3, took 54 seconds @ 284 MB/s > > With pre-fetch enabled (vfs.zfs.prefetch_disable=0): > > == Stripe Balanced (default) == > Read 15360MB using bs: 1048576, readers: 3, took 91 seconds @ 168 MB/s > == Load Balanced (zfslinux) == > Read 15360MB using bs: 1048576, readers: 3, took 108 seconds @ 142 MB/s > == Load Balanced (locality freebsd) == > Read 15360MB using bs: 1048576, readers: 3, took 48 seconds @ 320 MB/s > > In addition to the performance changes the code was also restructured, with > the help of Justin Gibbs, to provide a more logical flow which also ensures > vdevs loads are only calculated from the set of valid candidates. > > The following additional sysctls where added to allow the administrator > to tune the behaviour of the load algorithm: > * vfs.zfs.vdev.mirror.rotating_inc > * vfs.zfs.vdev.mirror.rotating_seek_inc > * vfs.zfs.vdev.mirror.rotating_seek_offset > * vfs.zfs.vdev.mirror.non_rotating_inc > * vfs.zfs.vdev.mirror.non_rotating_seek_inc > > These changes where based on work started by the zfsonlinux developers: > https://github.com/zfsonlinux/zfs/pull/1487 > > Reviewed by: gibbs, mav, will > MFC after: 2 weeks > Sponsored by: Multiplay No MFC yet.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140907160149.GC72244>