From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 6 18:07:45 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54F4F1065675; Mon, 6 Feb 2012 18:07:45 +0000 (UTC) (envelope-from tijl@coosemans.org) Received: from mailrelay006.isp.belgacom.be (mailrelay006.isp.belgacom.be [195.238.6.172]) by mx1.freebsd.org (Postfix) with ESMTP id 9FC5F8FC15; Mon, 6 Feb 2012 18:07:44 +0000 (UTC) X-Belgacom-Dynamic: yes X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjwGAFkPME9bsaFX/2dsb2JhbABCFq1xgUCBBoFyAQEFViMQCxgVGTkeiBgGsFCLLgEqBgELAQgFAwMJCIMRDixtBA4CCgIVDIMcBKgE Received: from 87.161-177-91.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([91.177.161.87]) by relay.skynet.be with ESMTP; 06 Feb 2012 18:37:52 +0100 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.14.5/8.14.5) with ESMTP id q16Hbpu9004031; Mon, 6 Feb 2012 18:37:51 +0100 (CET) (envelope-from tijl@coosemans.org) From: Tijl Coosemans To: freebsd-hackers@freebsd.org Date: Mon, 6 Feb 2012 18:37:42 +0100 User-Agent: KMail/1.13.7 (FreeBSD/10.0-CURRENT; KDE/4.7.3; i386; ; ) References: <4F2F7B7F.40508@FreeBSD.org> <20120206160136.GA35918@freebsd.org> <4F2FFFDA.2080608@FreeBSD.org> In-Reply-To: <4F2FFFDA.2080608@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart3534682.XvlJkZjjs3"; protocol="application/pgp-signature"; micalg=pgp-sha256 Content-Transfer-Encoding: 7bit Message-Id: <201202061837.48946.tijl@coosemans.org> Cc: Alexander Best , Alexander Motin Subject: Re: [RFT][patch] Scheduling for HTT and not only X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2012 18:07:45 -0000 --nextPart3534682.XvlJkZjjs3 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: quoted-printable On Monday 06 February 2012 17:29:14 Alexander Motin wrote: > On 02/06/12 18:01, Alexander Best wrote: >> On Mon Feb 6 12, Alexander Motin wrote: >>> I've analyzed scheduler behavior and think found the problem with HTT. >>> SCHED_ULE knows about HTT and when doing load balancing once a second, >>> it does right things. Unluckily, if some other thread gets in the way, >>> process can be easily pushed out to another CPU, where it will stay for >>> another second because of CPU affinity, possibly sharing physical core >>> with something else without need. >>> >>> I've made a patch, reworking SCHED_ULE affinity code, to fix that: >>> http://people.freebsd.org/~mav/sched.htt.patch >>> >>> This patch does three things: >>> - Disables strict affinity optimization when HTT detected to let more >>> sophisticated code to take into account load of other logical core(s). >>> - Adds affinity support to the sched_lowest() function to prefer >>> specified (last used) CPU (and CPU groups it belongs to) in case of >>> equal load. Previous code always selected first valid CPU of evens. It >>> caused threads migration to lower CPUs without need. >>> - If current CPU group has no CPU where the process with its priority >>> can run now, sequentially check parent CPU groups before doing global >>> search. That should improve affinity for the next cache levels. >>> >>> Who wants to do independent testing to verify my results or do some more >>> interesting benchmarks? :) >> >> i don't have any benchmarks to offer, but i'm seeing a massive increase = in >> responsiveness with your patch. with an unpatched kernel, opening xterm = while >> unrar'ing some huge archive could take up to 3 minutes!!! with your patc= h the >> time it takes for xterm to start is never> 10 seconds!!! >=20 > Thank you for the report. I can suggest explanation for this. Original=20 > code does only one pass looking for CPU where the thread can run=20 > immediately. That pass limited to the first level of CPU topology (for=20 > HTT systems it is one physical core). If it sees no good candidate, it=20 > just looks for the CPU with minimal load, ignoring thread priority. I=20 > suppose that may lead to priority violation, scheduling thread to CPU=20 > where higher-priority thread is running, where it may wait for a very=20 > long time, while there is some other CPU with minimal priority thread.=20 > My patch does more searches, that allows to handle priorities better. But why would unrar have a higher priority? --nextPart3534682.XvlJkZjjs3 Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iF4EABEIAAYFAk8wD+wACgkQfoCS2CCgtitlmgD/S3GpDkuBSUaNHdLgEEWFSowd KmHf7B6TAS3U3SVx6A0A/36+J5BUzFG/S/1eG6LNknsf/VeCzV96nrza27YLmi9c =gx99 -----END PGP SIGNATURE----- --nextPart3534682.XvlJkZjjs3--