From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 26 17:24:22 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E12310658B6 for ; Thu, 26 Feb 2009 17:24:22 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from el-out-1112.google.com (el-out-1112.google.com [209.85.162.183]) by mx1.freebsd.org (Postfix) with ESMTP id DB74C8FC19 for ; Thu, 26 Feb 2009 17:24:21 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: by el-out-1112.google.com with SMTP id r27so719915ele.13 for ; Thu, 26 Feb 2009 09:24:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=LJcXcEEBT5r3FCVtwwYnfvh8EeAtMLsQYBvF6ZDsVkA=; b=pyNEt1bP1WTKOxDtMq30Nbnrgk/IyQOxqh5gLz0KQxdWaQ+8X479zx9lnezggngB4W Cqh9tjwBL8Bl1tbq210eALUtlhXTnCjc7iYc/9+RDoavcn9w0Z01Wslm1GZqLCLXMHVS A4Kd9/D8me/eR6QBcaabfJtjmcGgXSxkrs9o0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=i2j6LSGbTm44PEkBNABG09m88o759cukoqjiquMR0nK4A064HF3hBfb2quODDGWWvp 47rQ2YEP6s1TW6TBsb5nBF+7/9RImQ5fY/4op6ANjuAQP5GfJnGcDxYNT47RaPI1zJtp y2iciWjidmOqwVR7w2+nuJRg/AFPUvdHLARCU= MIME-Version: 1.0 Received: by 10.90.94.15 with SMTP id r15mr797811agb.19.1235669061182; Thu, 26 Feb 2009 09:24:21 -0800 (PST) In-Reply-To: <49A6CF27.3000203@freebsd.org> References: <49A5D6FC.1090800@freebsd.org> <49A6CF27.3000203@freebsd.org> Date: Thu, 26 Feb 2009 09:24:20 -0800 Message-ID: <7d6fde3d0902260924r45ebb7c8i46cd6daf43a8171d@mail.gmail.com> From: Garrett Cooper To: Tim Kientzle Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Siddharth Prakash Singh , Robert Watson , freebsd-hackers@freebsd.org Subject: Re: Google SoC 2009 Idea X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2009 17:24:24 -0000 On Thu, Feb 26, 2009 at 9:19 AM, Tim Kientzle wrote: > Robert Watson wrote: >> >> On Wed, 25 Feb 2009, Tim Kientzle wrote: >> >>>> I have not gone through the process scheduler code of Free BSD. Hence,= I >>>> am not yet aware about the current support for Multicore Architectures= . >>> >>> Since you posted to a lot of different lists, I think you probably don'= t >>> already use FreeBSD. (If you did, why would you post to NetBSD and >>> DragonflyBSD lists?) =A0Scheduler work is quite complex and interacts h= eavily >>> with the rest of the system; it may not be a good choice for someone wh= o >>> doesn't already have a lot of experience with FreeBSD. >> >> All the things you say are true, but let's not be too hard on the new gu= y, >> however -- many of our GSoC students don't have previous FreeBSD >> kernel-hacking experience. =A0However, it does mean that they have to pi= ck >> project ideas that are well-suited to a significant warmup and investiga= tion >> period on the front end of the project. > > I apologize to Siddharth and others if I came off overly > harsh. =A0My intention was to caution him that he should > plan for a lot of work prior to GSoC if he wants to > tackle something that's at the core of the OS like this. > >> I'm also not convinced that a scheduler project along these lines would = be >> the most successful, but I wonder if a more experimental-spin proposal f= or >> looking at how to investigate poor scheduling decisions using dtrace, >> instrumentation and metrics to help us understand performance on NUMA >> systems, and exploring the impact of heuristics might go a long way. > > That's a good idea. =A0The thing that's always impressed > me about scheduling work is how very difficult it is to > test. =A0It's easy to change the scheduler code; it's > much harder to measure whether those changes have > made the scheduler better or not. > > Some testing support would help. =A0Ideally, something > non-intrusive that could be easily run on a lot > of different machines so as to collect better information > about the impacts of scheduler changes: > =A0* Load balancing: =A0How effectively are all cores being used? > =A0* CPU switching: =A0What percentage of the time does a thread > stay on the same core? > =A0* NUMA statistics: =A0How often does a thread get scheduled on a diffe= rent > processor from it's allocated memory? > =A0* Priority inversion: =A0How often is a higher-priority thread > idle while a lower-priority thread is running? > > A student who built such a tool and then ran some tests > with a variety of hardware and workloads could really > do a lot to advance scheduler development. =A0Eventually, > turning such a tool into something that anyone could run > and upload data to a central collection site could be > a huge advance. Speaking from experience, this is the way to go. If you don't do this you'll end up producing a potential black hole in terms of time and resources, which doesn't help your reputation on the project. Some food for thought. Cheers, -Garrett