From owner-svn-src-all@freebsd.org Mon Jan 27 20:43:36 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4BE6E22E686; Mon, 27 Jan 2020 20:43:36 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (glebi.us [162.251.186.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4861tJ06Klz4B1H; Mon, 27 Jan 2020 20:43:35 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id 00RKhXmB037372 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 27 Jan 2020 12:43:34 -0800 (PST) (envelope-from glebius@freebsd.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id 00RKhX5H037371; Mon, 27 Jan 2020 12:43:33 -0800 (PST) (envelope-from glebius@freebsd.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@freebsd.org using -f Date: Mon, 27 Jan 2020 12:43:33 -0800 From: Gleb Smirnoff To: John Baldwin Cc: Jeff Roberson , Ryan Stone , src-committers , svn-src-all , svn-src-head Subject: Re: svn commit: r357051 - head/sys/dev/bge Message-ID: <20200127204333.GX1268@FreeBSD.org> References: <202001231636.00NGawrr080128@repo.freebsd.org> <20200123230546.GG1268@FreeBSD.org> <20200124012458.GI1268@FreeBSD.org> <20200124024356.GK1268@FreeBSD.org> <20200124033243.GL1268@FreeBSD.org> <6a5485eb-8623-3316-0385-b0cda1624b05@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6a5485eb-8623-3316-0385-b0cda1624b05@FreeBSD.org> User-Agent: Mutt/1.12.2 (2019-09-21) X-Rspamd-Queue-Id: 4861tJ06Klz4B1H X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-1.47 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-0.68)[-0.679,0]; NEURAL_HAM_LONG(-0.79)[-0.787,0]; ASN(0.00)[asn:27348, ipnet:162.251.186.0/24, country:US] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jan 2020 20:43:36 -0000 On Sun, Jan 26, 2020 at 09:48:42AM +0000, John Baldwin wrote: J> > We also are concerned about that theoretically. Haven't yet seen effect J> > in practice, but our sessions are mostly longer living. First we have the J> > tunable to limit batching. Second, there are some ideas on how to improve J> > the garbage collector performance if it becomes an issue. J> J> There are other workloads than Netflix. ;) Verisign has incredibly short-lived J> connections with very high turnover. I think though that they have already J> abandoned the in-tree network stack for a userland stack built on netmap. Still, J> I think that there are probably other FreeBSD users that are probably somewhere J> in the middle that shouldn't be ignored. I understand that. First, my change doesn't create extra work for the garbage collector, but it potentially may affect its burstiness. If a machine has very high connection turnover it already may exhibit some problems with the PCB garbage collector on 12.0-RELEASE. Have we observed this yet? Note that PCBs are very different to other things protected by the network epoch: address lists, interface lists, firewall rules, etc. They are short lived. We got several ideas on how to deal with this potential problem: 1) The current PCB destructor does lots of things like freeing and unrefcounting associated multicast options, credentials, etc. This is because it was converted to epoch with a principle of minimal diff in r335015. However, since all operations with a PCB happen under its individual lock, we can free it differently. We can mark it INP_FREED and do all the destruction immediately, leaving only actual free to the garbage collector. Once this is achieved, the garbage collection can be batched at level of UMA. Jeff is working on that. 2) Use separate epoch for PCBs, leaving the network epoch for long lived structures only. Don't hold the PCB epoch long. 3) Just don't use epoch for PCBs. Decompose the hash lock into per slot hash lock. -- Gleb Smirnoff