Date: Sat, 15 Feb 2014 12:22:03 -0800 From: Brendan Gregg <brendan.gregg@joyent.com> To: Andriy Gapon <avg@freebsd.org> Cc: freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: l2arc_feed_thread cpu utlization Message-ID: <CA%2BXzFFhCF__qTNAiUVz6fpzWygVJBWp4_5UkQkWPYToD8dFPKQ@mail.gmail.com> In-Reply-To: <52FF566D.3060601@FreeBSD.org> References: <52B2D8D6.8090306@FreeBSD.org> <52FE0378.7070608@FreeBSD.org> <CA%2BXzFFgN1Mq94ydeA=F1uy3oA=OPiF0w4yrWr4smXY9ovR%2Bqdg@mail.gmail.com> <52FF566D.3060601@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
G'Day Andriy,
On Sat, Feb 15, 2014 at 3:58 AM, Andriy Gapon <avg@freebsd.org> wrote:
> on 14/02/2014 22:23 Brendan Gregg said the following:
> > G'Day Andriy,
> >
> > Thanks for the patch. If most of the data is in one list (anyone have
> statistics
> > to confirm such a likelyhood? I know this happened a lot
> pre-list-split), then I
> > think this means we only scan that at 1/32nd of the previous rate. It
> should
> > solve the CPU issue, but could make warmup very slow.
>
> Brendan,
>
> I do not have any stats, but I think that the data should be spread more
> or less
> evenly between the lists. I mean the 16 sub-lists for data and 16
> sub-lists for
> metadata. First, a list is picked up based on hash and that _should_
> produce
> more or less even distribution. Second, if the hash funciton is not good
> enough
> then whole list splitting is pointless.
> In either case this was just a quick hack on my part.
>
Ah, I'm sorry, I should have read more of the code earlier; I had assumed
the split
algorithm was something else, and I'm wrong. It should be even.
So, based on get_buf_info(), I think we can DTrace how buf_hash() is mapped
to
the lists to get an idea of the distribution. Eg (on illumos, which has the
same
buf_hash() code):
# dtrace -n 'fbt::buf_hash:return { @ = lquantize(arg1 & (32 - 1), 0, 32,
1); } tick-30s { exit(0); }'
dtrace: description 'fbt::buf_hash:return ' matched 2 probes
CPU ID FUNCTION:NAME
7 30 :tick-30s
value ------------- Distribution ------------- count
< 0 | 0
0 |@@ 20581
1 |@ 12578
2 |@ 6004
3 |@@ 15215
4 |@ 4660
5 | 2952
6 | 3678
7 |@@ 14091
8 | 2402
9 |@@ 20998
10 |@ 5805
11 |@ 6564
12 |@@@@ 35560
13 |@ 13021
14 | 4348
15 |@@ 17035
16 |@@ 15406
17 |@ 5512
18 |@ 13222
19 |@ 5488
20 |@ 5404
21 |@@ 13583
22 |@ 7453
23 |@ 4794
24 | 3738
25 |@@@ 24918
26 |@@ 15566
27 |@ 5324
28 |@ 12112
29 |@@ 13966
30 |@@ 16668
31 |@ 9904
>= 32 | 0
So that looks reasonably even - every bucket is in use. I think your patch
should be good.
Brendan
--
Brendan Gregg, Joyent http://dtrace.org/blogs/brendan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2BXzFFhCF__qTNAiUVz6fpzWygVJBWp4_5UkQkWPYToD8dFPKQ>