Date: Mon, 27 Jan 2020 12:43:33 -0800 From: Gleb Smirnoff <glebius@freebsd.org> To: John Baldwin <jhb@freebsd.org> Cc: Jeff Roberson <jroberson@jroberson.net>, Ryan Stone <rysto32@gmail.com>, src-committers <src-committers@freebsd.org>, svn-src-all <svn-src-all@freebsd.org>, svn-src-head <svn-src-head@freebsd.org> Subject: Re: svn commit: r357051 - head/sys/dev/bge Message-ID: <20200127204333.GX1268@FreeBSD.org> In-Reply-To: <6a5485eb-8623-3316-0385-b0cda1624b05@FreeBSD.org> References: <202001231636.00NGawrr080128@repo.freebsd.org> <CAFMmRNyP5nKPZ62c9yRktH4t23GAgnpc2u8kHhUevUCu3doAQQ@mail.gmail.com> <20200123230546.GG1268@FreeBSD.org> <CAFMmRNwhcV-%2BSsq_a1RZJYXC8wvnUDLDxdFeHD7PgL5wuierkg@mail.gmail.com> <20200124012458.GI1268@FreeBSD.org> <CAFMmRNz3TVPfP7_fhBMa=gdN%2BM4OdCco3z1qO=mPOm57-QOsng@mail.gmail.com> <20200124024356.GK1268@FreeBSD.org> <alpine.BSF.2.21.9999.2001231657400.1198@desktop> <20200124033243.GL1268@FreeBSD.org> <6a5485eb-8623-3316-0385-b0cda1624b05@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Jan 26, 2020 at 09:48:42AM +0000, John Baldwin wrote: J> > We also are concerned about that theoretically. Haven't yet seen effect J> > in practice, but our sessions are mostly longer living. First we have the J> > tunable to limit batching. Second, there are some ideas on how to improve J> > the garbage collector performance if it becomes an issue. J> J> There are other workloads than Netflix. ;) Verisign has incredibly short-lived J> connections with very high turnover. I think though that they have already J> abandoned the in-tree network stack for a userland stack built on netmap. Still, J> I think that there are probably other FreeBSD users that are probably somewhere J> in the middle that shouldn't be ignored. I understand that. First, my change doesn't create extra work for the garbage collector, but it potentially may affect its burstiness. If a machine has very high connection turnover it already may exhibit some problems with the PCB garbage collector on 12.0-RELEASE. Have we observed this yet? Note that PCBs are very different to other things protected by the network epoch: address lists, interface lists, firewall rules, etc. They are short lived. We got several ideas on how to deal with this potential problem: 1) The current PCB destructor does lots of things like freeing and unrefcounting associated multicast options, credentials, etc. This is because it was converted to epoch with a principle of minimal diff in r335015. However, since all operations with a PCB happen under its individual lock, we can free it differently. We can mark it INP_FREED and do all the destruction immediately, leaving only actual free to the garbage collector. Once this is achieved, the garbage collection can be batched at level of UMA. Jeff is working on that. 2) Use separate epoch for PCBs, leaving the network epoch for long lived structures only. Don't hold the PCB epoch long. 3) Just don't use epoch for PCBs. Decompose the hash lock into per slot hash lock. -- Gleb Smirnoff
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20200127204333.GX1268>