From owner-freebsd-performance@FreeBSD.ORG Mon Nov 5 15:00:13 2007 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C6FF16A417 for ; Mon, 5 Nov 2007 15:00:13 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from fallbackmx02.syd.optusnet.com.au (fallbackmx02.syd.optusnet.com.au [211.29.133.72]) by mx1.freebsd.org (Postfix) with ESMTP id B341613C4C5 for ; Mon, 5 Nov 2007 15:00:12 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail02.syd.optusnet.com.au (mail02.syd.optusnet.com.au [211.29.132.183]) by fallbackmx02.syd.optusnet.com.au (8.12.11.20060308/8.12.11) with ESMTP id lA56twJU020144 for ; Mon, 5 Nov 2007 17:55:58 +1100 Received: from c211-30-219-213.carlnfd3.nsw.optusnet.com.au (c211-30-219-213.carlnfd3.nsw.optusnet.com.au [211.30.219.213]) by mail02.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id lA56sg0E011570 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 5 Nov 2007 17:54:47 +1100 Date: Mon, 5 Nov 2007 17:54:41 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Jeff Roberson In-Reply-To: <20071104132540.S544@10.0.0.1> Message-ID: <20071105171941.C19146@delplex.bde.org> References: <1698.10.202.77.103.1194211291.squirrel@webmail.superhero.nl> <20071104132540.S544@10.0.0.1> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-performance@freebsd.org, "Gelsema, P \(Patrick\) - FreeBSD" Subject: Re: ULE vs 4BSD in RELENG_7 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2007 15:00:13 -0000 On Sun, 4 Nov 2007, Jeff Roberson wrote: > On Sun, 4 Nov 2007, Gelsema, P (Patrick) - FreeBSD wrote: >> w/o patch >> hulk# time make -j8 buildkernel >> 837.808u 138.167s 10:28.96 155.1% 6349+1349k 2873+7780io 303pf+0w >> >> w patch >> hulk# time make -j8 buildkernel >> 838.554u 168.316s 10:52.10 154.4% 6263+1332k 6489+7791io 11pf+0w >> >> I only understand the 10:52 seconds thing, but it looks like it's >> stressing less and still getting things done. > > Well this doesn't look very good at all. System time increased by 30 > seconds! Must be too many extra context switches. The minimum slice value > is probably too small and you've got an average of 4 threads per-core. 838 seconds for a kernel build! I'm unhappy with ~32 seconds for a kernel build on a system with 1/2 or 1/4 as much CPU here, with ULE still taking an extra 0.5 seconds. Mine is not an equivalent kernel build since it is for ~4.10 with ~5.2 userland (so that the benchmark isn't affected by anything except the kernel). 4BSD: "make depend": 8.12 real 5.09 user 1.34 sys "make": 31.94 real 56.18 user 4.49 sys ULE: "make depend": 8.09 real 5.16 user 1.19 sys "make": 32.12 real 55.72 user 4.91 sys The kernel is about 2/3 as large as GENERIC, with no acpi and no modules. The machine is a Turion X2 2GHz in i386 mode. The buffer cache was 100% warm. All makes normal ones (no buildkernel or modules pessimizations) with -j4. Before recent vm changes, the real times were about 0.20 seconds smaller for "make". Building on a local file system gives real times about 2 seconds smaller for "make". Building a ~5.2 kernel on a local file system on this machine takes about 10 seconds longer (~42 seconds for 4BSD and ~43 seconds for ULE). Building a ~5.2 world over nfs (with a cold cache) on this machine takes about 825 seconds with 4BSD and 833 seconds with ULE. -current building itself is much slower due to source bloat and gcc bloat: For build a -current kernel with the same features on an Opteron X2 1.8GHz in amd64 mode over an untuned nfs with an untuned (INVARIANTS) kernel: 4BSD (several months ago), gcc-3.4.6: "make depend": 110.32 real 15.83 user 10.20 sys "make": 86.43 real 128.17 user 19.05 sys ULE, gcc-4.2.1: "make depend": 83.34 real 15.39 user 8.67 sys "make": 108.03 real 170.77 user 23.83 sys The "make depend" times are not directly comparable. The untuned nfs is so slow that "make -j4 depend" takes 50-100 seconds no matter how many kernels are built in parallel, due to dead time waiting for nfs, so I build 6 kernels in parallel ("make depend" itself is mostly serial) on the slow machine but not normally. The "make" times are directly comparable (they are for "make -j4" of a single kernel on 2-way SMP systems). Bruce