Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 26 Apr 2014 13:29:43 +0200
From:      Jilles Tjoelker <jilles@stack.nl>
To:        Karl Pielorz <kpielorz_lst@tdx.co.uk>
Cc:        freebsd-hackers@freebsd.org
Subject:   Re: HAST performance / dtrace / _umtx_op
Message-ID:  <20140426112943.GA81447@stack.nl>
In-Reply-To: <B044E9557B63332185C7792F@study64.tdx.co.uk>
References:  <B044E9557B63332185C7792F@study64.tdx.co.uk>

next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Apr 26, 2014 at 11:30:25AM +0100, Karl Pielorz wrote:
> I've been looking at HAST recently - and noticed there's a huge
> performance hit (on 10-STABLE anyway) even if you setup a disk with a
> remote of 'none' - switching from the 'raw' disk to '/dev/hast/disk'
> shows a 50% performance hit [in testing having an active secondary via
> 10Gbit link doesn't seem to show any further performance drop].

> Thinking of digging deeper I setup a single disk HAST system (with no
> remote node) and ran dtrace on the daemon handling that disk. I then
> DD'ed 1Gbyte of data to that disk.

> The output from dtrace shows:

> "
> Elapsed Times for PID 5219,
> 
>          SYSCALL          TIME (ns)
>            pread            4439902
>           pwrite         7617448542
>            ioctl        11370282332
>     sigtimedwait        20015976362
>         _umtx_op        36514993910
> "

> To my untrained eye that looks like it spent more time on locking than
> anything else? - Followed by ioctl's - then, considerably less time on
> pread/pwrite (which must be the bit that's copying the data)

> Presumably the time spent in sigtimedwait would have actually been
> 'waiting' time - i.e. not active CPU usage kind of time?

> Does that seem to be the case? - At least then I'll hopefully have
> some pointer where to look further for why there's such a performance
> hit.

_umtx_op implements blocking not only for mutexes but also for condition
variables, and in the latter case waiting for a long time does not
indicate a problem.

-- 
Jilles Tjoelker



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140426112943.GA81447>