From owner-freebsd-java@FreeBSD.ORG Tue Sep 18 14:16:13 2007 Return-Path: Delivered-To: freebsd-java@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC04E16A46C; Tue, 18 Sep 2007 14:16:13 +0000 (UTC) (envelope-from kurt@intricatesoftware.com) Received: from mail1.intricatesoftware.com (cl-18.ewr-01.us.sixxs.net [IPv6:2001:4830:1200:11::2]) by mx1.freebsd.org (Postfix) with ESMTP id 5DFC213C4A8; Tue, 18 Sep 2007 14:16:13 +0000 (UTC) (envelope-from kurt@intricatesoftware.com) Received: from seraph.intricatesoftware.com (relay@localhost.intricatesoftware.com [IPv6:::1]) by mail1.intricatesoftware.com (8.14.1/8.13.4) with ESMTP id l8IEG8Z1000316; Tue, 18 Sep 2007 10:16:08 -0400 (EDT) Received: from seraph.intricatesoftware.com (truk@localhost.intricatesoftware.com [127.0.0.1]) by seraph.intricatesoftware.com (8.14.1/8.14.1) with ESMTP id l8IEG9SG030255; Tue, 18 Sep 2007 10:16:09 -0400 (EDT) Message-ID: <46EFDDA8.2030603@intricatesoftware.com> Date: Tue, 18 Sep 2007 10:16:08 -0400 From: Kurt Miller User-Agent: Thunderbird 2.0.0.5 (X11/20070804) MIME-Version: 1.0 To: Daniel Eischen References: <46EC1B55.4060202@FreeBSD.org> <46EDDCC9.90403@intricatesoftware.com> <200709180721.48995.kurt@intricatesoftware.com> In-Reply-To: X-Enigmail-Version: 0.95.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SMTP-Vilter-Version: 1.3.6 X-SMTP-Vilter-Virus-Backend: clamd X-SMTP-Vilter-Status: clean X-SMTP-Vilter-clamd-Virus-Status: clean X-Spamd-Symbols: ALL_TRUSTED X-SMTP-Vilter-Spam-Backend: spamd X-Spam-Score: -1.4 X-Spam-Threshold: 5.0 X-Spam-Probability: -0.3 X-Its-A-Nuisance: This is spam X-SMTP-Vilter-Unwanted-Backend: attachment X-SMTP-Vilter-attachment-Unwanted-Status: clean Cc: Kip Macy , Kris Kennaway , performance@freebsd.org, freebsd-java@freebsd.org Subject: Re: Massive performance loss from OS::sleep hack X-BeenThere: freebsd-java@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting Java to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2007 14:16:13 -0000 Hi Daniel, Daniel Eischen wrote: > On Tue, 18 Sep 2007, Kurt Miller wrote: > >> David Xu confirmed for me that pthread_yield() does give some >> time to lower priority threads on 7.0 using thr. Attached and inline >> are two patches for the 1.5 port that is how I suggest the issue be >> addressed. > > I don't think you should rely on pthread_yield() doing that, > it is dependent on the scheduler, and if you are running > in SCHED_FIFO/SCHED_RR, it won't work with libthr either. > Currently, you can only run in SCHED_FIFO or SCHED_RR as > root using libthr, but that hopefully won't always be > true. Shoot I forgot to mention a few things in my last email... The c test program I resurrected from 18 months ago had pthread_attr_setschedpolicy(&attr, SCHED_FIFO); in it. The jdk doesn't set the schedpolicy it uses the default one. That line was left over from my testing different policies under kse to see if I could work- around the problem. Since the jdk doesn't set the policy, I believe we don't need to be concerned with SCHED_FIFO and SCHED_RR. > I think the way that will always work is to lower the > priority and raise it back again after the yield, when > using thread priorities. This should work for libthr, > libc_r, and libkse under all versions of the OS. On the surface this sounded promising, but I can envision at least one case where temporarily lowering a thread priority will not work very well. Take three threads A, B & C each with different priorities: A highest, C lowest. Let's say A is in a busy loop calling Thread.Yield(). B is in a busy loop also calling Thread.Yield(). C is doing something that needs some time slices every so often. A calls Thread.Yield(), lowers priority to C's level, B gets some time next, it calls Thread.Yield(), lowers priority to C's level. A, B, C are all at equal level now. Perhaps C will get some time first, or B, or A. Suppose the order is C first, B next. A will only get a time slice if B continues to call yield and by chance the system schedules A before B. The behavior becomes non-deterministic. The worst case is A is starved. Also to make matters a bit more complicated there are non-java threads to contend with too. There could be some non-java locking involved as well. It would require an investment of someone else's time to work all the potential issues out. > As for knobs, it would be nice to be able to turn off > the default lowering and raising of priorities for > Thread.yield() even when using thread priorities. Most > likely, the only applications that relies on that behavior > are the TCK tests themselves. I certainly wouldn't expect > Thread.yield() to allow a lower priority thread to > run. Considering Sun's reluctance to correct the TCK test, I would venture a guess that there are applications out there that rely on Thread.Yield() to give time to lower priority threads (in-spite of the significant amount of Java books, white papers, certification tests, etc that say not to do that). > Are you allowed to supply knobs when running the TCK > tests? Or does the JVM have to do the right thing by > default? The JVM gets certified on the default threading library for the standard options. Generally the non-standard options (-Xfoo) are not certified. Attempting to certify the JVM for multiple threading libraries, os versions, architectures, and options would be cost prohibitive. I believe what I've suggested provides a solution to the performance issue (at the expense of supporting thread priorities on < 7.0), while not introducing unexpected or non-certifiable behavior. Motivated people on this list should file bug reports w/Sun against the Thread.Yield() API. It is about time Sun clarify the expected behavior of of Thread.Yield() with respect to thread priorities. At a minimum the current description of Thread.Yield() is in conflict with JSR-133: JavaTM Memory Model and Thread Specification, Section 12: "The Java specification does not guarantee preemptive multithreading or any kind of fairness guarantee. There is no hard guarantee that any thread will surrender the CPU and allow other threads to be scheduled." Not to mention the 10 years of non-official publications that say not to rely on Thread.Yield() to do anything portable at all. Regards, -Kurt