From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 3 09:12:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C2371065670; Sat, 3 Mar 2012 09:12:33 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id D47EF8FC08; Sat, 3 Mar 2012 09:12:32 +0000 (UTC) Received: by bkcjc3 with SMTP id jc3so2822934bkc.13 for ; Sat, 03 Mar 2012 01:12:31 -0800 (PST) Received-SPF: pass (google.com: domain of mavbsd@gmail.com designates 10.205.137.14 as permitted sender) client-ip=10.205.137.14; Authentication-Results: mr.google.com; spf=pass (google.com: domain of mavbsd@gmail.com designates 10.205.137.14 as permitted sender) smtp.mail=mavbsd@gmail.com; dkim=pass header.i=mavbsd@gmail.com Received: from mr.google.com ([10.205.137.14]) by 10.205.137.14 with SMTP id im14mr6984109bkc.137.1330765951773 (num_hops = 1); Sat, 03 Mar 2012 01:12:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=7INbNAJqjvMeOrXa/mcKQVZRkTJ7JduxqxgPGzlP1BE=; b=Nl+P90vXTOIO7Y69O0lWvWIyh5JJaFZthN+H6KsimBofBJIxA40ZmYLUZrSNd7HQ8R ji1DWrTqUiCHYZmO5RAlJcdPrl8JAg8OhkIESDSbikM2N5NzJwwYl5S/u8mr/U2MGY+V Rvauq7zfPOQNpYEgB2T6mafpCvgSqDrFXfOsYxFlMHU5HNH9KHOEhd8JTf8oIvWSXPNv w23IR8A9sl9esVV7tI8/j7A2gf/BTzHMO00bJDvCOCxE0eukGotLcx7kmgJn7hEyEXC8 AFR3pKXK1uU7N+lcmZTKSpTtUyCPkToAypTJHthSK3oyRamS4lbxRsdmBr1FPtQh4FLu QTGQ== Received: by 10.205.137.14 with SMTP id im14mr5577412bkc.137.1330765951649; Sat, 03 Mar 2012 01:12:31 -0800 (PST) Received: from mavbook2.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id y9sm13758658bkw.5.2012.03.03.01.12.29 (version=SSLv3 cipher=OTHER); Sat, 03 Mar 2012 01:12:30 -0800 (PST) Sender: Alexander Motin Message-ID: <4F51E07C.4020706@FreeBSD.org> Date: Sat, 03 Mar 2012 11:12:28 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.2) Gecko/20120226 Thunderbird/10.0.2 MIME-Version: 1.0 To: Adrian Chadd References: <4F2F7B7F.40508@FreeBSD.org> <4F366E8F.9060207@FreeBSD.org> <4F367965.6000602@FreeBSD.org> <4F396B24.5090602@FreeBSD.org> <4F3978BC.6090608@FreeBSD.org> <4F3990EA.1080002@FreeBSD.org> <4F3C0BB9.6050101@FreeBSD.org> <4F3E807A.60103@FreeBSD.org> <4F3E8858.4000001@FreeBSD.org> <4F4ACF2C.50300@m5p.com> <4F4B67AB.40907@m5p.com> <4F4C17E2.2040101@m5p.com> <4F516281.30603@m5p.com> <4F51CAE9.20905@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, George Mitchell Subject: Re: [RFT][patch] Scheduling for HTT and not only X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2012 09:12:33 -0000 On 03/03/12 10:59, Adrian Chadd wrote: > Right. Is this written up in a PR somewhere explaining the problem in > as much depth has you just have? Have no idea. I am new at this area and haven't looked on PRs yet. > And thanks for this, it's great to see some further explanation of the > current issues the scheduler faces. By the way I've just reproduced the problem with compilation. On dual-core system net/mpd5 compilation in one stream takes 17 seconds. But with two low-priority non-interactive CPU-burning threads running it takes 127 seconds. I'll try to analyze it more now. I have feeling that there could be more factors causing priority violation than I've described below. > On 2 March 2012 23:40, Alexander Motin wrote: >> On 03/03/12 05:24, Adrian Chadd wrote: >>> >>> mav@, can you please take a look at George's traces and see if there's >>> anything obviously silly going on? >>> He's reporting that your ULE work hasn't improved his (very) degenerate >>> case. >> >> >> As I can see, my patch has nothing to do with the problem. My patch improves >> SMP load balancing, while in this case problem is different. In some cases, >> when not all CPUs are busy, my patch could mask the problem by using more >> CPUs, but not in this case when dnets consumes all available CPUs. >> >> I still not feel very comfortable with ULE math, but as I understand, in >> both illustrated cases there is a conflict between clearly CPU-bound dnets >> threads, that consume all available CPU and never do voluntary context >> switches, and more or less interactive other threads. If other threads >> detected to be "interactive" in ULE terms, they should preempt dnets threads >> and everything will be fine. But "batch" (in ULE terms) threads never >> preempt each other, switching context only about 10 times per second, as >> hardcoded in sched_slice variable. Kernel build by definition consumes too >> much CPU time to be marked "interactive". exo-helper-1 thread in >> interact.out could potentially be marked "interactive", but possibly once it >> consumed some CPU to become "batch", it is difficult for it to get back, as >> waiting in a runq is not counted as sleep and each time it is getting >> running, it has some new work to do, so it remains "batch". May be if CPU >> time accounting was more precise it would work better (by accounting those >> short periods when threads really sleeps voluntary), but not with present >> sampled logic with 1ms granularity. As result, while dnets threads each time >> consume full 100ms time slices, other threads are starving, getting running >> only 10 times per second to voluntary switch out in just a few milliseconds. >> >> >>> On 2 March 2012 16:14, George Mitchell wrote: >>>> >>>> On 03/02/12 18:06, Adrian Chadd wrote: >>>>> >>>>> >>>>> Hi George, >>>>> >>>>> Have you thought about providing schedgraph traces with your >>>>> particular workload? >>>>> >>>>> I'm sure that'll help out the scheduler hackers quite a bit. >>>>> >>>>> THanks, >>>>> >>>>> >>>>> Adrian >>>>> >>>> >>>> I posted a couple back in December but I haven't created any more >>>> recently: >>>> >>>> http://www.m5p.com/~george/ktr-ule-problem.out >>>> http://www.m5p.com/~george/ktr-ule-interact.out >>>> >>>> To the best of my knowledge, no one ever examined them. -- George >> >> >> >> -- >> Alexander Motin >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Alexander Motin