Date: Fri, 3 Jul 2020 13:30:22 -0600 From: Alan Somers <asomers@freebsd.org> To: freebsd-geom@freebsd.org Subject: Single-threaded bottleneck in geli Message-ID: <CAOtMX2hFaNCmwkuhfWWqNwACETtomnJroTC1_giOO_iFj0SKFQ@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
I'm using geli, gmultipath, and ZFS on a large system, with hundreds of
drives. What I'm seeing is that under at least some workloads, the overall
performance is limited by the single geom kernel process. procstat and
kgdb aren't much help in telling exactly why this process is using so much
CPU, but it certainly must be related to the fact that over 15,000 IOPs are
going through that thread. What can I do to improve this situation? Would
it make sense to enable direct dispatch for geli? That would hurt
single-threaded performance, but probably improve performance for highly
multithreaded workloads like mine.
Example top output:
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
13 root -8 - 0B 96K CPU46 46 82.7H 70.54%
geom{g_down}
13 root -8 - 0B 96K - 9 35.5H 25.32%
geom{g_up}
-Alan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2hFaNCmwkuhfWWqNwACETtomnJroTC1_giOO_iFj0SKFQ>