Date: Wed, 25 Oct 2006 17:43:34 +0200 From: Michel Talon <talon@lpthe.jussieu.fr> To: freebsd-stable@FreeBSD.ORG, guido@gvr.org Subject: Re: Gmirror performanc Message-ID: <20061025154334.GA45269@lpthe.jussieu.fr>
next in thread | raw e-mail | index | archive | help
Oliver Fromme <olli@lurza.secnetix.de> wrote:
> > I tried with -b split -s <various sizes>, -b round-robin, -b load.
> > (dd-ing as done with a bs of 1m; I see the transaction size is 128Kb,
> > unless the split method is used, in which case the transaction size
> > gies down. When round-robin is used, the transaction size is 128Kb/s,
> > but the number of transaction per second goes down.).
> >
> > I cannot explain why I should not get a higher read speed. Anyone?
>
> dd is a sequential, single-threaded operation, so it will
> only use one disk at a time. It's not really suitable as
> a benchmark for real-world things.
I see the same problem as Guido, that is gmirror on two disks is not faster
that each disk separately and is even markedly slower. I disagree with your
explanation, which moreover contradicts the definition of split, round-robin,
etc. in the man page of gmirror. Experimentally, observing the disk throughput
via iostat shows that both disks are involved in the IO.
asmodee# dd if=/dev/mirror/gms1a of=/dev/null bs=256k
556+0 records in
556+0 records out
145752064 bytes transferred in 8.785839 secs (16589431 bytes/sec)
(note the low throughput, the disks are individually able to do more than 20
MB/s, this is on an old machine), while iostat shows:
asmodee% iostat -w 1
tty ad0 ad4 cpu
tin tout KB/t tps MB/s KB/t tps MB/s us ni sy in id
0 60 61.38 116 6.95 61.40 117 7.01 0 0 1 0 99
0 60 64.00 129 8.06 64.00 128 7.99 0 0 2 1 97
0 60 64.00 128 7.99 64.00 129 8.06 0 0 2 1 97
0 60 64.00 129 8.06 64.00 129 8.06 0 0 1 1 98
0 60 63.50 128 7.93 63.50 127 7.87 0 0 5 2 94
that is the transaction is evenly distributed on both disks ad0 and ad4
(which are on 2 separate channels). The problem is that each disk works at
only 8MB/s while it is able of 3 times more.
Looking at the kernel driver /usr/src/sys/geom/mirror/g_mirror.c, it seems
that the load is split on the various disks in the function
g_mirror_request_split() in a way which is simulatneous for all providers.
How is it that after that the request proceeds so slowly, i don't know. But
i doubt very much it will be any different wether you have a "real world load"
or a simple dd.
--
Michel TALON
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061025154334.GA45269>