From owner-freebsd-hackers@freebsd.org Wed Mar 29 20:00:43 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67FD1D24F5F for ; Wed, 29 Mar 2017 20:00:43 +0000 (UTC) (envelope-from vangyzen@FreeBSD.org) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [199.48.133.146]) by mx1.freebsd.org (Postfix) with ESMTP id 50EE4C49 for ; Wed, 29 Mar 2017 20:00:42 +0000 (UTC) (envelope-from vangyzen@FreeBSD.org) Received: from sweettea.beer.town (unknown [76.164.8.130]) by smtp.vangyzen.net (Postfix) with ESMTPSA id C748D5646B for ; Wed, 29 Mar 2017 15:00:36 -0500 (CDT) From: Eric van Gyzen Subject: One Priority Per Run Queue To: freebsd-hackers@freebsd.org Message-ID: <1aafd6a2-828c-06f5-bdac-e4c953a403b5@FreeBSD.org> Date: Wed, 29 Mar 2017 15:00:36 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Mar 2017 20:00:43 -0000 The FreeBSD schedulers assign four priorities to each run queue, making those priorities effectively equal. This breaks POSIX real-time priorities. Applications that use real-time scheduling use sched_get_priority_min() and sched_get_priority_max() [0] to determine the available range of priorities, and then use simple arithmetic to assign relatively higher or lower priorities. If an application configures two threads with priorities MAX and MAX-1 (for example), POSIX says the thread at priority MAX must be chosen if it is runnable. Since our implementation puts these two priorities in the same run queue, it may choose either thread, so it does not conform. The above functions currently return 0 and 31, respectively. One solution would change max() to return 7 and change other code to translate the 8 POSIX values into the 32 FreeBSD values. However, this would also not conform, because "conforming implementations shall provide a priority range of at least 32 priorities for this policy." [1] I propose that we assign one priority per run queue: https://reviews.freebsd.org/D10188 This would conform to POSIX. On a certain commercial block storage product, this change made no difference in performance. Benchmarks of buildworld on two different machines actually showed a tiny improvement in performance. [2] Please test the above change, especially if you have an interesting workload that might be sensitive to scheduler behavior. If you already know this change would cause problems, please point me toward the details. Assigning 4 priorities per run queue also caused a recent portability issue in ZFS, although that was fixed by r314058. Eric [0] http://pubs.opengroup.org/onlinepubs/9699919799/functions/sched_get_priority_max.html [1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_08_04_01 [2] http://www.vangyzen.net/FreeBSD/1ppq/1ppq.html