From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 3 07:40:31 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55D381065672; Sat, 3 Mar 2012 07:40:31 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 90E7A8FC0C; Sat, 3 Mar 2012 07:40:30 +0000 (UTC) Received: by bkcjc3 with SMTP id jc3so2789617bkc.13 for ; Fri, 02 Mar 2012 23:40:29 -0800 (PST) Received-SPF: pass (google.com: domain of mavbsd@gmail.com designates 10.204.133.220 as permitted sender) client-ip=10.204.133.220; Authentication-Results: mr.google.com; spf=pass (google.com: domain of mavbsd@gmail.com designates 10.204.133.220 as permitted sender) smtp.mail=mavbsd@gmail.com; dkim=pass header.i=mavbsd@gmail.com Received: from mr.google.com ([10.204.133.220]) by 10.204.133.220 with SMTP id g28mr6959241bkt.16.1330760429435 (num_hops = 1); Fri, 02 Mar 2012 23:40:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=/yX6nf4sL8posMq+kQ5BjfpdNhnYLTJBDqV5mPL2gIk=; b=QQh4ggJr/itm51fP3Q7neW4wxH0qcPKrz+d3+hwdOo4s2T9NQXKQ6oULKzsIH4PaYT 2NBmljy3FoSv0TcHGF7ukj0LmKOFfDovWX2lKK7xoLJu4upf6AGe7iecvi5FGQzFSwlY BEpmEGB4EBiFHKNeFyOqi7zMOkIgeGO7YZElHiTZvcwbmCvHwOnSWsEsMgHFtPYgiJyi 44rvcegUlWkZDDRlpYz84V0QOsCcWGUqAPXaqrlJ8jObH5VhzuHA4kqdzCFUTSA9lcht wFmcii1Y4pzYGMnS3JQrL8HuW0OAmaqI25yIU8+hPP6rVVXa2tt2npXVcscmz1MMjsLv GDhg== Received: by 10.204.133.220 with SMTP id g28mr5584876bkt.16.1330760427889; Fri, 02 Mar 2012 23:40:27 -0800 (PST) Received: from mavbook2.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id jd17sm13440420bkb.4.2012.03.02.23.40.26 (version=SSLv3 cipher=OTHER); Fri, 02 Mar 2012 23:40:27 -0800 (PST) Sender: Alexander Motin Message-ID: <4F51CAE9.20905@FreeBSD.org> Date: Sat, 03 Mar 2012 09:40:25 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.2) Gecko/20120226 Thunderbird/10.0.2 MIME-Version: 1.0 To: Adrian Chadd References: <4F2F7B7F.40508@FreeBSD.org> <4F366E8F.9060207@FreeBSD.org> <4F367965.6000602@FreeBSD.org> <4F396B24.5090602@FreeBSD.org> <4F3978BC.6090608@FreeBSD.org> <4F3990EA.1080002@FreeBSD.org> <4F3C0BB9.6050101@FreeBSD.org> <4F3E807A.60103@FreeBSD.org> <4F3E8858.4000001@FreeBSD.org> <4F4ACF2C.50300@m5p.com> <4F4B67AB.40907@m5p.com> <4F4C17E2.2040101@m5p.com> <4F516281.30603@m5p.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, George Mitchell Subject: Re: [RFT][patch] Scheduling for HTT and not only X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2012 07:40:31 -0000 Hi. On 03/03/12 05:24, Adrian Chadd wrote: > mav@, can you please take a look at George's traces and see if there's > anything obviously silly going on? > He's reporting that your ULE work hasn't improved his (very) degenerate case. As I can see, my patch has nothing to do with the problem. My patch improves SMP load balancing, while in this case problem is different. In some cases, when not all CPUs are busy, my patch could mask the problem by using more CPUs, but not in this case when dnets consumes all available CPUs. I still not feel very comfortable with ULE math, but as I understand, in both illustrated cases there is a conflict between clearly CPU-bound dnets threads, that consume all available CPU and never do voluntary context switches, and more or less interactive other threads. If other threads detected to be "interactive" in ULE terms, they should preempt dnets threads and everything will be fine. But "batch" (in ULE terms) threads never preempt each other, switching context only about 10 times per second, as hardcoded in sched_slice variable. Kernel build by definition consumes too much CPU time to be marked "interactive". exo-helper-1 thread in interact.out could potentially be marked "interactive", but possibly once it consumed some CPU to become "batch", it is difficult for it to get back, as waiting in a runq is not counted as sleep and each time it is getting running, it has some new work to do, so it remains "batch". May be if CPU time accounting was more precise it would work better (by accounting those short periods when threads really sleeps voluntary), but not with present sampled logic with 1ms granularity. As result, while dnets threads each time consume full 100ms time slices, other threads are starving, getting running only 10 times per second to voluntary switch out in just a few milliseconds. > On 2 March 2012 16:14, George Mitchell wrote: >> On 03/02/12 18:06, Adrian Chadd wrote: >>> >>> Hi George, >>> >>> Have you thought about providing schedgraph traces with your >>> particular workload? >>> >>> I'm sure that'll help out the scheduler hackers quite a bit. >>> >>> THanks, >>> >>> >>> Adrian >>> >> >> I posted a couple back in December but I haven't created any more >> recently: >> >> http://www.m5p.com/~george/ktr-ule-problem.out >> http://www.m5p.com/~george/ktr-ule-interact.out >> >> To the best of my knowledge, no one ever examined them. -- George -- Alexander Motin