Date: Fri, 3 Jul 2020 14:22:11 -0600 From: Alan Somers <asomers@freebsd.org> To: =?UTF-8?Q?Pawe=C5=82_Jakub_Dawidek?= <pawel@dawidek.net> Cc: freebsd-geom@freebsd.org Subject: Re: Single-threaded bottleneck in geli Message-ID: <CAOtMX2hHaEzOT0jmc_QcukVZjRKUtCm55bTT9Q5=BNCcL9rf%2Bg@mail.gmail.com> In-Reply-To: <48AB9191-5984-4BE0-BF98-7548424D06BF@dawidek.net> References: <CAOtMX2hFaNCmwkuhfWWqNwACETtomnJroTC1_giOO_iFj0SKFQ@mail.gmail.com> <48AB9191-5984-4BE0-BF98-7548424D06BF@dawidek.net>
next in thread | previous in thread | raw e-mail | index | archive | help
I don't. What I meant was that a single thread (geom) is limiting the
performance of the system overall. I'm certain, based on top, gstat, and
zpool iostat, that geom is the limiting factor on this system.
-Alan
On Fri, Jul 3, 2020 at 2:18 PM Pawe=C5=82 Jakub Dawidek <pawel@dawidek.net>
wrote:
> Hi Alan,
>
> why do you think it will hurt single-threaded performance?
>
> --
> Pawe=C5=82 Jakub Dawidek
>
>
>
> > On Jul 3, 2020, at 12:30, Alan Somers <asomers@freebsd.org> wrote:
> >
> > =EF=BB=BFI'm using geli, gmultipath, and ZFS on a large system, with hu=
ndreds of
> > drives. What I'm seeing is that under at least some workloads, the
> overall
> > performance is limited by the single geom kernel process. procstat and
> > kgdb aren't much help in telling exactly why this process is using so
> much
> > CPU, but it certainly must be related to the fact that over 15,000 IOPs
> are
> > going through that thread. What can I do to improve this situation?
> Would
> > it make sense to enable direct dispatch for geli? That would hurt
> > single-threaded performance, but probably improve performance for highl=
y
> > multithreaded workloads like mine.
> >
> > Example top output:
> > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMA=
ND
> > 13 root -8 - 0B 96K CPU46 46 82.7H 70.54%
> > geom{g_down}
> > 13 root -8 - 0B 96K - 9 35.5H 25.32%
> > geom{g_up}
> >
> > -Alan
> > _______________________________________________
> > freebsd-geom@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-geom
> > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org"
>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2hHaEzOT0jmc_QcukVZjRKUtCm55bTT9Q5=BNCcL9rf%2Bg>