From owner-freebsd-arch@FreeBSD.ORG Wed Jul 6 14:49:56 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3102E106564A for ; Wed, 6 Jul 2011 14:49:56 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 865718FC08 for ; Wed, 6 Jul 2011 14:49:55 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA01286 for ; Wed, 06 Jul 2011 17:49:54 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E147611.6060100@FreeBSD.org> Date: Wed, 06 Jul 2011 17:49:53 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: SCHED_ULE preempt_thresh: PRI_MIN_KERN -> PRI_MIN_IDLE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 14:49:56 -0000 I do not have sufficient knowledge of SCHED_ULE, so maybe I shouldn't even talk about this, but I couldn't help but notice that many (many) users have reported in the past heavy interactivity problems with SCHED_ULE under high load, especially I/O-related load. The universal advice has always been to tune preempt_thresh via sysctl kern.sched.preempt_thresh=224. I think that David Xu was the first person that I saw recommending this. In all cases users have reported significant improvements. I must add that I also have the experience and I do use preempt_thresh=224 to this day. Now, I would like to discuss this phenomenon in two veins: 1. Why do we see the interactivity problem with the default setting of preempt_thresh=PRI_MIN_KERN (provided that PREEMPTION is enabled and FULL_PREEMPTION is not)? Could this be a general ULE issue? Or could it be because of some particular hogs (like, purely hypothetically speaking, GEOM threads)? 2. Why don't we change the default (for PREEMPTION and !FULL_PREEMPTION case) to preempt_thresh=PRI_MIN_IDLE? Plus sides of this have been reported via anecdotes. What down sides could there be? Unfortunately somehow I just couldn't grasp ULE priorities and preemption, so I'd like to ask for help of those who already have understood these things. Thank you. -- Andriy Gapon