From owner-freebsd-current@freebsd.org Mon May 29 12:20:03 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31B32CFDC09 for ; Mon, 29 May 2017 12:20:03 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DE67077209; Mon, 29 May 2017 12:20:01 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA14950; Mon, 29 May 2017 15:19:59 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1dFJeQ-000MT6-TK; Mon, 29 May 2017 15:19:58 +0300 Subject: Re: NFS client perf. degradation when SCHED_ULE is used (was when SMP enabled) To: Rick Macklem Cc: "cem@freebsd.org" , "jeff@freebsd.org" , Ryan Stone , FreeBSD Current References: From: Andriy Gapon Message-ID: Date: Mon, 29 May 2017 15:19:22 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 May 2017 12:20:03 -0000 On 28/05/2017 01:20, Rick Macklem wrote: > - with the "obvious change" mentioned in r312426's commit message, using > (flags & SW_TYPE_MASK) == SWT_RELINQUISH instead of (flag & SWT_RELINQUISH) > 121minutes Rick, can I see how exactly your variant of the obvious change looks in your version? I am asking, because I meant applying that change to the original code while it can be also interpreted as applying it to the code after r312426. That is, does it looks like this: preempted = !((td->td_flags & TDF_SLICEEND) || ((flags & SW_TYPE_MASK) == SWT_RELINQUISH)); or like this: preempted = !(td->td_flags & TDF_SLICEEND) && ((flags & SW_TYPE_MASK) == SWT_RELINQUISH); > I also tested: > ((flags & SW_PREEMPT) != 0 || (flags & SW_TYPE_MASK) == SWT_IDLE || > (flags & SW_TYPE_MASK) == SWT_IWAIT) > and it also resulted in 121minutes So, this sets the preempted flag for SWT_IDLE and SWT_IWAIT cases. The flag makes any difference only if the current thread is calling mi_switch() but remains running (of which typical cases are preemption and yielding). As far as I can tell, mi_switch(SWT_IWAIT) is only called when the thread is already inhibited via TD_SET_IWAIT, so that should not make any difference. SWT_IDLE is set only when the current thread is an idle thread, so that should not make any difference either. Thus, I am puzzled as to why this change could make any difference. Could you please post full code snippets for each local change that you tried? Also, could you please capture KTR sched trace while running the test on the kernel with no local modifications and on the 1yr old kernel? Ideally, I would like to see the trace with KTR_SCHED | KTR_RUNQ compiled into the kernel via KTR_COMPILE and then enabled at the run time via debug.ktr.mask=0x20400000. Thank you. -- Andriy Gapon