From owner-freebsd-geom@freebsd.org  Fri Jul  3 20:18:23 2020
Return-Path: <owner-freebsd-geom@freebsd.org>
Delivered-To: freebsd-geom@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9AAA53582E6
 for <freebsd-geom@mailman.nyi.freebsd.org>;
 Fri,  3 Jul 2020 20:18:23 +0000 (UTC)
 (envelope-from pawel@dawidek.net)
Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72])
 by mx1.freebsd.org (Postfix) with ESMTP id 49z5rH2Cb8z3SHS;
 Fri,  3 Jul 2020 20:18:22 +0000 (UTC)
 (envelope-from pawel@dawidek.net)
Received: from [192.168.250.156] (c-67-169-9-63.hsd1.ca.comcast.net
 [67.169.9.63])
 by mail.dawidek.net (Postfix) with ESMTPSA id 22FE8445;
 Fri,  3 Jul 2020 22:18:16 +0200 (CEST)
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (1.0)
Subject: Re: Single-threaded bottleneck in geli
From: =?utf-8?Q?Pawe=C5=82_Jakub_Dawidek?= <pawel@dawidek.net>
In-Reply-To: <CAOtMX2hFaNCmwkuhfWWqNwACETtomnJroTC1_giOO_iFj0SKFQ@mail.gmail.com>
Date: Fri, 3 Jul 2020 13:18:13 -0700
Cc: freebsd-geom@freebsd.org
Message-Id: <48AB9191-5984-4BE0-BF98-7548424D06BF@dawidek.net>
References: <CAOtMX2hFaNCmwkuhfWWqNwACETtomnJroTC1_giOO_iFj0SKFQ@mail.gmail.com>
To: Alan Somers <asomers@freebsd.org>
X-Mailer: iPhone Mail (17F80)
X-Rspamd-Queue-Id: 49z5rH2Cb8z3SHS
X-Spamd-Bar: ----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[];
 ASN(0.00)[asn:16276, ipnet:91.121.0.0/16, country:FR]
X-BeenThere: freebsd-geom@freebsd.org
X-Mailman-Version: 2.1.33
Precedence: list
List-Id: GEOM-specific discussions and implementations
 <freebsd-geom.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-geom>,
 <mailto:freebsd-geom-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-geom/>
List-Post: <mailto:freebsd-geom@freebsd.org>
List-Help: <mailto:freebsd-geom-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-geom>,
 <mailto:freebsd-geom-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 03 Jul 2020 20:18:23 -0000

Hi Alan,

why do you think it will hurt single-threaded performance?

--=20
Pawe=C5=82 Jakub Dawidek



> On Jul 3, 2020, at 12:30, Alan Somers <asomers@freebsd.org> wrote:
>=20
> =EF=BB=BFI'm using geli, gmultipath, and ZFS on a large system, with hundr=
eds of
> drives.  What I'm seeing is that under at least some workloads, the overal=
l
> performance is limited by the single geom kernel process.  procstat and
> kgdb aren't much help in telling exactly why this process is using so much=

> CPU, but it certainly must be related to the fact that over 15,000 IOPs ar=
e
> going through that thread.  What can I do to improve this situation?  Woul=
d
> it make sense to enable direct dispatch for geli?  That would hurt
> single-threaded performance, but probably improve performance for highly
> multithreaded workloads like mine.
>=20
> Example top output:
>  PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
>   13 root         -8    -     0B    96K CPU46   46  82.7H  70.54%
> geom{g_down}
>   13 root         -8    -     0B    96K -        9  35.5H  25.32%
> geom{g_up}
>=20
> -Alan
> _______________________________________________
> freebsd-geom@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-geom
> To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org"