From owner-freebsd-hackers@freebsd.org Sun Apr 9 22:10:01 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24872D3683F for ; Sun, 9 Apr 2017 22:10:01 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 16E07A4C for ; Sun, 9 Apr 2017 22:10:01 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from Alfreds-MacBook-Pro-2.local (unknown [IPv6:2601:645:8003:a4d6:80a8:3cdd:4e29:76fd]) by elvis.mu.org (Postfix) with ESMTPSA id B19E9346DDF5 for ; Sun, 9 Apr 2017 15:10:00 -0700 (PDT) Subject: Re: One Priority Per Run Queue To: freebsd-hackers@freebsd.org References: <1aafd6a2-828c-06f5-bdac-e4c953a403b5@FreeBSD.org> From: Alfred Perlstein Organization: FreeBSD Message-ID: <836da108-7e25-fc94-2c84-bc2f85bb6398@freebsd.org> Date: Sun, 9 Apr 2017 15:10:33 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Apr 2017 22:10:01 -0000 On 3/29/17 2:18 PM, Warner Losh wrote: > On Wed, Mar 29, 2017 at 2:00 PM, Eric van Gyzen wrote: >> The FreeBSD schedulers assign four priorities to each run queue, making >> those priorities effectively equal. This breaks POSIX real-time priorities. >> >> Applications that use real-time scheduling use sched_get_priority_min() >> and sched_get_priority_max() [0] to determine the available range of >> priorities, and then use simple arithmetic to assign relatively higher >> or lower priorities. If an application configures two threads with >> priorities MAX and MAX-1 (for example), POSIX says the thread at >> priority MAX must be chosen if it is runnable. Since our implementation >> puts these two priorities in the same run queue, it may choose either >> thread, so it does not conform. >> >> The above functions currently return 0 and 31, respectively. One >> solution would change max() to return 7 and change other code to >> translate the 8 POSIX values into the 32 FreeBSD values. However, this >> would also not conform, because "conforming implementations shall >> provide a priority range of at least 32 priorities for this policy." [1] >> >> I propose that we assign one priority per run queue: >> >> https://reviews.freebsd.org/D10188 >> >> This would conform to POSIX. On a certain commercial block storage >> product, this change made no difference in performance. Benchmarks of >> buildworld on two different machines actually showed a tiny improvement >> in performance. [2] >> >> Please test the above change, especially if you have an interesting >> workload that might be sensitive to scheduler behavior. If you already >> know this change would cause problems, please point me toward the details. >> >> Assigning 4 priorities per run queue also caused a recent portability >> issue in ZFS, although that was fixed by r314058. > How does this scheme prevent starvation of low priority processes? Or > rather, how will this change after this change. > It would seem that for userland this should allow for starvation as that's the point. However once inside the kernel and any locks are taken you must do at minimum priority lending or bump priority higher otherwise you can cause deadlock. I thought we already do this...? -Alfred