From owner-svn-src-all@freebsd.org Fri Jan 24 20:23:39 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 818621FB1B3; Fri, 24 Jan 2020 20:23:39 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4849Zf2vCMz4WB6; Fri, 24 Jan 2020 20:23:38 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id v5UNiagC7nCigv5UOiW0Fl; Fri, 24 Jan 2020 13:23:37 -0700 X-Authority-Analysis: v=2.3 cv=cZisUULM c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=kj9zAlcOel0A:10 a=Jdjhy38mL1oA:10 a=ndaoGXS1AAAA:8 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=3RbmEhYP8JuRmH6FH_UA:9 a=YozzxExVAemzLD1i:21 a=50F0wAOjQmEiZ6MV:21 a=CjuIK1q_8ugA:10 a=mFeOnlTyF09QQMGr2mMI:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from slippy.cwsent.com (slippy8 [10.2.2.6]) by spqr.komquats.com (Postfix) with ESMTPS id AB2FF2A8; Fri, 24 Jan 2020 12:23:34 -0800 (PST) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id 00OKNYE2094992; Fri, 24 Jan 2020 12:23:34 -0800 (PST) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id 00OKNXlS094989; Fri, 24 Jan 2020 12:23:34 -0800 (PST) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <202001242023.00OKNXlS094989@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Hans Petter Selasky cc: Gleb Smirnoff , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r357004 - in head/sys: kern sys In-reply-to: <7d7db96d-26b1-1d2b-9f8d-a3f8fbe8c33c@selasky.org> References: <202001230124.00N1OlXi029506@repo.freebsd.org> <23f272a4-c997-a454-19d6-10392713e71f@selasky.org> <20200124170532.GO1268@FreeBSD.org> <7d7db96d-26b1-1d2b-9f8d-a3f8fbe8c33c@selasky.org> Comments: In-reply-to Hans Petter Selasky message dated "Fri, 24 Jan 2020 20:21:54 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 24 Jan 2020 12:23:33 -0800 X-CMAE-Envelope: MS4wfOObIeG0RL9rH4h8D7a/Ov/cNaRVE9/0nSVYTjuNjub0+RvCIKu6/H+5lhzpHLl9IP4XVxaZ1g8YocdTxm3Qgd7lQKuBISvMfirWXa8ceUymrhANQCIO fxWkRiL3k3d4QcdXxESA+2jQWCMz7sTJlCFwgEo6d1yQUxkqdWldS54l9/yRmhss4DgEvg3ytMkKM616XmT6E+W4dVGrmx5/n5Kk7mc7o7Ik6cQW9wD8dZoa uePtX/mxMcaheBYgZuJ6WzifXeflHCA+/f5PMdVifPhh3us2nlyW+x1wWGH+1RhgPtfRDo4mgpVM2zFo6Y5lJw== X-Rspamd-Queue-Id: 4849Zf2vCMz4WB6 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 64.59.134.13) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-4.15 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCPT_COUNT_FIVE(0.00)[5]; REPLYTO_EQ_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[13.134.59.64.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-2.45)[ip: (-6.53), ipnet: 64.59.128.0/20(-3.17), asn: 6327(-2.46), country: CA(-0.09)]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2020 20:23:39 -0000 In message <7d7db96d-26b1-1d2b-9f8d-a3f8fbe8c33c@selasky.org>, Hans Petter Sela sky writes: > On 2020-01-24 18:05, Gleb Smirnoff wrote: > > On Fri, Jan 24, 2020 at 10:24:53AM +0100, Hans Petter Selasky wrote: > > H> What you want to do here is right, but how it is implemented is wrong, > > H> in my opinion. > > H> > > H> 1) Remove intr_epoch_batch. Most network drivers use interrupt > > H> moderation, and a timeout of 1000 iterations can easily become 1 second > > H> under heavly load ! > > > > If a driver has interrupt moderation than epoch batching counter > > basically won't ever grow over 1. It kicks in only of driver doesn't > > have it, or receives interrupts at a very high rate. > > Depending on the load an interrupt imposes, this batching counter may > cause epochs to last for a long time. Have you considered using ticks > for this instead? I.E. if the congestion lasts more than two ticks, then > re-acquire the EPOCH? > > For example if the network controller spends more time processing > packets then there is between interrupts, then this unconditional 1000x > thing can be quite dangerous. > > > > > H> 2) You need to make a new request function for interrupts which take a > > H> pointer to an EPOCH and replace that IH_NET in hflags! > > > > Initially I did that way, but then pondered over this approach and have > > abandoned it. Most likely we will have just few globally recognized > > epochs in the kernel. And even less might be associated with interrupt > > handlers. Complexity and performance impact of using a pointer are > > noted by Drew in D23347. > > Let not the network epoch become the new Giant of EPOCH's. There might > be realtime constraints for EPOCH's aswell. I also had that concern yesterday. Obtaining an EPOCH higher up the call stack or lower down depends on the workload. Are there any measurements that moving the EPOCHs higher in the call stack improves one workload (and configuration) or another? Could one type of workload or configuration benefit from this at the expense of another workload or configuration? > > Secondly, I don't see how 8000 IRQs per second will cause any problem > accessing a pointer in the interrupt handler structure? A well designed > ethernet driver use coalescing of packets and the the IRQ rate goes > down. What Drew is referring to are badly designed ethernet cards with > very small DMA rings, so the speak. > > If the IRQ handler is executed hundreds of thousand times per second, > that is a problem in itself and reading one more pointer will have > little impact on the system. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few.