From owner-freebsd-stable@freebsd.org  Thu Sep 22 09:40:52 2016
Return-Path: <owner-freebsd-stable@freebsd.org>
Delivered-To: freebsd-stable@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5556BE43DF
 for <freebsd-stable@mailman.ysv.freebsd.org>;
 Thu, 22 Sep 2016 09:40:52 +0000 (UTC) (envelope-from slw@zxy.spb.ru)
Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 6733DB0C;
 Thu, 22 Sep 2016 09:40:52 +0000 (UTC) (envelope-from slw@zxy.spb.ru)
Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD))
 (envelope-from <slw@zxy.spb.ru>)
 id 1bn0Us-000IRP-9v; Thu, 22 Sep 2016 12:40:50 +0300
Date: Thu, 22 Sep 2016 12:40:50 +0300
From: Slawa Olhovchenkov <slw@zxy.spb.ru>
To: Julien Charbon <jch@freebsd.org>
Cc: Konstantin Belousov <kostikbel@gmail.com>, freebsd-stable@FreeBSD.org,
 hiren panchasara <hiren@strugglingcoder.info>
Subject: Re: 11.0 stuck on high network load
Message-ID: <20160922094050.GA2840@zxy.spb.ru>
References: <20160916181839.GC2960@zxy.spb.ru>
 <20160916183053.GL9397@strugglingcoder.info>
 <20160916190330.GG2840@zxy.spb.ru>
 <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org>
 <20160919204328.GN2840@zxy.spb.ru>
 <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org>
 <20160920202633.GQ2840@zxy.spb.ru>
 <f644cd52-4377-aa90-123a-3a2887972bbc@freebsd.org>
 <20160921083148.GU2840@zxy.spb.ru>
 <05ba1a3a-2d99-f8e2-40a1-4c1fca317db3@freebsd.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <05ba1a3a-2d99-f8e2-40a1-4c1fca317db3@freebsd.org>
User-Agent: Mutt/1.5.24 (2015-08-30)
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: slw@zxy.spb.ru
X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-stable>, 
 <mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable/>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
 <mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Sep 2016 09:40:52 -0000

On Thu, Sep 22, 2016 at 11:28:38AM +0200, Julien Charbon wrote:

> >>> What purpose to not skip locked tcptw in this loop?
> >>
> >>  If I understand your question correctly:  According to your pmcstat
> >> result, tcp_tw_2msl_scan() currently struggles with a write lock
> >> (__rw_wlock_hard) and the only write lock used tcp_tw_2msl_scan() is
> >> INP_WLOCK.  No sign of contention on TW_RLOCK(V_tw_lock) currently.
> > 
> > As I see in code, tcp_tw_2msl_scan got first node from V_twq_2msl and
> > need got RW lock on inp w/o alternates. Can tcp_tw_2msl_scan skip current node
> > and go to next node in V_twq_2msl list if current node locked by some
> > reasson?
> 
>  Interesting question indeed:  It is not optimal that all simultaneous
> calls to tcp_tw_2msl_scan() compete for the same oldest tcptw.  The next
> tcptws in the list are certainly old enough also.
> 
>  Let me see if I can make a simple change that makes kernel threads
> calling tcp_tw_2msl_scan() at same time to work on a different old
> enough tcptws.  So far, I found only solutions quite complex to implement.

Simple solution is skip in each thread ncpu elemnts and skip curent
cpu number elements at start, if I understund you correctly.