From owner-freebsd-arch@FreeBSD.ORG Sat Nov 18 18:17:31 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5793816A8B7 for ; Sat, 18 Nov 2006 18:17:31 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2-3.pacific.net.au [61.8.2.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7866D43F09 for ; Sat, 18 Nov 2006 18:16:11 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout2.pacific.net.au (Postfix) with ESMTP id 9021E6E100 for ; Sun, 19 Nov 2006 05:16:13 +1100 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id 5CD6127413 for ; Sun, 19 Nov 2006 05:16:13 +1100 (EST) Date: Sun, 19 Nov 2006 05:16:12 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: arch@freebsd.org Message-ID: <20061119041421.I16763@delplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: What is the PREEMPTION option good for? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Nov 2006 18:17:31 -0000 PREEMPTION may be needed for correctness, including for very low latency switching to high priority tasks, but it is a pessimization for most things that I tried (mainly makeworld), and the details of the pessimizations indicate that it doesn't actually give correctness either. Makeworld in a certain SMP configuration here takes 853+-5 seconds without PREEMPTION or IPI_PREEMPTION, and 863+-5 seconds with PREEMPTION and IPI_PREEMPTION (both without KSE; KSE gives another pessimization in the 5-10 second range). Most of the difference is caused by pgzero becoming too active with PREEMPTION. The behaviour with PREEMPTION under SMP is similar to that under both UP and SMP when the PREEMPTION ifdef was first added to vm_zeroidle.c a couple of years ago. pgzero is active for about 3 times as long (18 seconds instead of 6 for my current makeworld benchmark. This reduces the reported system time by even more than the extra time spent in pagezero, but for some reason (probably cache thrashing, though pgzero uses nontemporal writes on the benchmark machine) it increases the real time by about the same amount as the extra time spent in pgzero. A couple of years ago, pgzero did this even for !SMP because it had a very broken priority so it rarely (never?) got preempted. Now its preemption is broken similarly under SMP with PREEMPTION but without IPI_PREEMPTION, since it takes an IPI to preempt it in many cases. Its code is: % for (;;) { % if (vm_page_zero_check()) { % vm_page_zero_idle(); % #ifndef PREEMPTION % if (sched_runnable()) { % mtx_lock_spin(&sched_lock); % mi_switch(SW_VOL, NULL); % mtx_unlock_spin(&sched_lock); % } % #endif without PREEMPTION, it yields voluntarily, and this works fine. With PREEMPTION and !SMP, it gets preempted, and this works not so fine (it has slightly higher overheads). With PREEMPTION and SMP and >1 CPU but no IPI_PREEMPTION, this cannot work, and even with IPI_PREEMPTION it doesn't work now in practice (IPI_PREEMPTION gives a small pessimization due to more context switches without significantly affecting the time spent in pgzero). If PREEMPTION should actually be best here, why doesn't the main idle thread depend on it? Does anyone have an example where PREEMPTION makes a useful difference? Bruce