From owner-freebsd-current@FreeBSD.ORG Wed Mar 14 15:09:06 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 183891065673 for ; Wed, 14 Mar 2012 15:09:06 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from qmta11.emeryville.ca.mail.comcast.net (qmta11.emeryville.ca.mail.comcast.net [76.96.27.211]) by mx1.freebsd.org (Postfix) with ESMTP id EBB698FC18 for ; Wed, 14 Mar 2012 15:09:05 +0000 (UTC) Received: from omta18.emeryville.ca.mail.comcast.net ([76.96.30.74]) by qmta11.emeryville.ca.mail.comcast.net with comcast id lSo21i0061bwxycABT8zUZ; Wed, 14 Mar 2012 15:08:59 +0000 Received: from damnhippie.dyndns.org ([24.8.232.202]) by omta18.emeryville.ca.mail.comcast.net with comcast id lT8x1i00k4NgCEG8eT8ydt; Wed, 14 Mar 2012 15:08:59 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q2EF8utS023740; Wed, 14 Mar 2012 09:08:56 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: "O. Hartmann" In-Reply-To: <4F60A167.7020609@mail.zedat.fu-berlin.de> References: <4F60A167.7020609@mail.zedat.fu-berlin.de> Content-Type: text/plain; charset="us-ascii" Date: Wed, 14 Mar 2012 09:08:56 -0600 Message-ID: <1331737736.1099.30.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Current FreeBSD Subject: Re: CURRENT: make -jX buildworld doesn't work X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Mar 2012 15:09:06 -0000 On Wed, 2012-03-14 at 14:47 +0100, O. Hartmann wrote: > This is no compalin, since make buildworld works with one thread. > > But I'd like to report a funny thing I witnessed. > > On two boxes equipted with Core2Dou CPUS (E8500, 2 cores/threads, > Q6600, 4 cores/threads) a parallel make buildworld works fine with the > most recent sources of FreeBSD 10.0-CURRENT/amd64 (both boxes have 8 GB > RAM, both boxes use a very close/similar setup and configuration). > > I moved in my lab towards a brand new Sandy-Bridge-E box with 32GB RAM > and the CPU is a Core i7-3930X with six cores/12 threads. On this box, > even "make buildworld" with -j2 fails to build. It builds fine with a > vanilla "make buildworld". > > Also funny is, that even with only one thread, the 3 GHz Core2Duo > methusalem systems "outperform" in compile time the 3,2 GHZ driven Intel > youngster. > > Maybe there is an issue in FreeBSD 10 with the TURBO BOOST? I'm a bit > time constraint now, but I'm willing to do some tests with advices from > the experts. > > Lets give you some informations I think it could be valuable. Please > request more, if this isn't sufficient. > > I realize/I'm aware that both hardware and OS are brandnew! But > hopefully this is something important and could be "fixed" - if my > observation is a real observation ... > > Regards, > Oliver > > KERNEL: > > # CPU frequency control > device cpufreq > device cpuctl > device nvram > device coretemp > > /etc/rc.conf: > performance_cx_lowest="HIGH" # Online CPU idle state > performance_cpu_freq="NONE" # Online CPU frequency > economy_cx_lowest="HIGH" # Offline CPU idle state > economy_cpu_freq="NONE" # Offline CPU frequency > > #> sysctl kern.timecounter > kern.timecounter.tc.i8254.mask: 65535 > kern.timecounter.tc.i8254.counter: 52702 > kern.timecounter.tc.i8254.frequency: 1193182 > kern.timecounter.tc.i8254.quality: 0 > kern.timecounter.tc.HPET.mask: 4294967295 > kern.timecounter.tc.HPET.counter: 2109995661 > kern.timecounter.tc.HPET.frequency: 14318180 > kern.timecounter.tc.HPET.quality: 950 > kern.timecounter.tc.ACPI-fast.mask: 16777215 > kern.timecounter.tc.ACPI-fast.counter: 3590445 > kern.timecounter.tc.ACPI-fast.frequency: 3579545 > kern.timecounter.tc.ACPI-fast.quality: 900 > kern.timecounter.tc.TSC-low.mask: 4294967295 > kern.timecounter.tc.TSC-low.counter: 1510440780 > kern.timecounter.tc.TSC-low.frequency: 12537463 > kern.timecounter.tc.TSC-low.quality: -100 > kern.timecounter.stepwarnings: 0 > kern.timecounter.hardware: HPET > kern.timecounter.choice: TSC-low(-100) ACPI-fast(900) HPET(950) i8254(0) > dummy(-1000000) > kern.timecounter.tick: 5 > kern.timecounter.invariant_tsc: 1 > kern.timecounter.smp_tsc: 0 > > # > #> dmesg | grep cpu > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 1 > cpu2 (AP): APIC ID: 2 > cpu3 (AP): APIC ID: 3 > cpu4 (AP): APIC ID: 4 > cpu5 (AP): APIC ID: 5 > cpu6 (AP): APIC ID: 6 > cpu7 (AP): APIC ID: 7 > cpu8 (AP): APIC ID: 8 > cpu9 (AP): APIC ID: 9 > cpu10 (AP): APIC ID: 10 > cpu11 (AP): APIC ID: 11 > cpu0: on acpi0 > cpu1: on acpi0 > cpu2: on acpi0 > cpu3: on acpi0 > cpu4: on acpi0 > cpu5: on acpi0 > cpu6: on acpi0 > cpu7: on acpi0 > cpu8: on acpi0 > cpu9: on acpi0 > cpu10: on acpi0 > cpu11: on acpi0 > coretemp0: on cpu0 > est0: on cpu0 > p4tcc0: on cpu0 > coretemp1: on cpu1 > est1: on cpu1 > p4tcc1: on cpu1 > coretemp2: on cpu2 > est2: on cpu2 > p4tcc2: on cpu2 > coretemp3: on cpu3 > est3: on cpu3 > p4tcc3: on cpu3 > coretemp4: on cpu4 > est4: on cpu4 > p4tcc4: on cpu4 > coretemp5: on cpu5 > est5: on cpu5 > p4tcc5: on cpu5 > coretemp6: on cpu6 > est6: on cpu6 > p4tcc6: on cpu6 > coretemp7: on cpu7 > est7: on cpu7 > p4tcc7: on cpu7 > coretemp8: on cpu8 > est8: on cpu8 > p4tcc8: on cpu8 > coretemp9: on cpu9 > est9: on cpu9 > p4tcc9: on cpu9 > coretemp10: on cpu10 > est10: on cpu10 > p4tcc10: on cpu10 > coretemp11: on cpu11 > est11: on cpu11 > p4tcc11: on cpu11 > > > > #> sysctl dev.cpu > dev.cpu.0.%desc: ACPI CPU > dev.cpu.0.%driver: cpu > dev.cpu.0.%location: handle=\_SB_.P000 > dev.cpu.0.%pnpinfo: _HID=none _UID=0 > dev.cpu.0.%parent: acpi0 > dev.cpu.0.coretemp.delta: 54 > dev.cpu.0.coretemp.resolution: 1 > dev.cpu.0.coretemp.tjmax: 91.0C > dev.cpu.0.coretemp.throttle_log: 0 > dev.cpu.0.temperature: 37.0C > dev.cpu.0.freq: 3200 > dev.cpu.0.freq_levels: 3201/130000 3200/130000 3000/119000 2800/109000 > 2600/99000 2400/89000 2200/80000 2000/71000 1800/62000 1600/54000 > 1400/46000 1225/40250 1200/39000 1050/34125 900/29250 750/24375 > 600/19500 450/14625 300/9750 150/4875 > dev.cpu.0.cx_supported: C1/1 > dev.cpu.0.cx_lowest: C1 > dev.cpu.0.cx_usage: 100.00% last 759us > dev.cpu.1.%desc: ACPI CPU > dev.cpu.1.%driver: cpu > dev.cpu.1.%location: handle=\_SB_.P001 > dev.cpu.1.%pnpinfo: _HID=none _UID=0 > dev.cpu.1.%parent: acpi0 > dev.cpu.1.coretemp.delta: 54 > dev.cpu.1.coretemp.resolution: 1 > dev.cpu.1.coretemp.tjmax: 91.0C > dev.cpu.1.coretemp.throttle_log: 0 > dev.cpu.1.temperature: 37.0C > dev.cpu.1.cx_supported: C1/1 > dev.cpu.1.cx_lowest: C1 > dev.cpu.1.cx_usage: 100.00% last 24782us > dev.cpu.2.%desc: ACPI CPU > dev.cpu.2.%driver: cpu > dev.cpu.2.%location: handle=\_SB_.P002 > dev.cpu.2.%pnpinfo: _HID=none _UID=0 > dev.cpu.2.%parent: acpi0 > dev.cpu.2.coretemp.delta: 53 > dev.cpu.2.coretemp.resolution: 1 > dev.cpu.2.coretemp.tjmax: 91.0C > dev.cpu.2.coretemp.throttle_log: 0 > dev.cpu.2.temperature: 38.0C > dev.cpu.2.cx_supported: C1/1 > dev.cpu.2.cx_lowest: C1 > dev.cpu.2.cx_usage: 100.00% last 23314us > dev.cpu.3.%desc: ACPI CPU > dev.cpu.3.%driver: cpu > dev.cpu.3.%location: handle=\_SB_.P003 > dev.cpu.3.%pnpinfo: _HID=none _UID=0 > dev.cpu.3.%parent: acpi0 > dev.cpu.3.coretemp.delta: 53 > dev.cpu.3.coretemp.resolution: 1 > dev.cpu.3.coretemp.tjmax: 91.0C > dev.cpu.3.coretemp.throttle_log: 0 > dev.cpu.3.temperature: 38.0C > dev.cpu.3.cx_supported: C1/1 > dev.cpu.3.cx_lowest: C1 > dev.cpu.3.cx_usage: 100.00% last 8973us > dev.cpu.4.%desc: ACPI CPU > dev.cpu.4.%driver: cpu > dev.cpu.4.%location: handle=\_SB_.P004 > dev.cpu.4.%pnpinfo: _HID=none _UID=0 > dev.cpu.4.%parent: acpi0 > dev.cpu.4.coretemp.delta: 50 > dev.cpu.4.coretemp.resolution: 1 > dev.cpu.4.coretemp.tjmax: 91.0C > dev.cpu.4.coretemp.throttle_log: 0 > dev.cpu.4.temperature: 41.0C > dev.cpu.4.cx_supported: C1/1 > dev.cpu.4.cx_lowest: C1 > dev.cpu.4.cx_usage: 100.00% last 9446us > dev.cpu.5.%desc: ACPI CPU > dev.cpu.5.%driver: cpu > dev.cpu.5.%location: handle=\_SB_.P005 > dev.cpu.5.%pnpinfo: _HID=none _UID=0 > dev.cpu.5.%parent: acpi0 > dev.cpu.5.coretemp.delta: 50 > dev.cpu.5.coretemp.resolution: 1 > dev.cpu.5.coretemp.tjmax: 91.0C > dev.cpu.5.coretemp.throttle_log: 0 > dev.cpu.5.temperature: 41.0C > dev.cpu.5.cx_supported: C1/1 > dev.cpu.5.cx_lowest: C1 > dev.cpu.5.cx_usage: 100.00% last 50156us > dev.cpu.6.%desc: ACPI CPU > dev.cpu.6.%driver: cpu > dev.cpu.6.%location: handle=\_SB_.P006 > dev.cpu.6.%pnpinfo: _HID=none _UID=0 > dev.cpu.6.%parent: acpi0 > dev.cpu.6.coretemp.delta: 48 > dev.cpu.6.coretemp.resolution: 1 > dev.cpu.6.coretemp.tjmax: 91.0C > dev.cpu.6.coretemp.throttle_log: 0 > dev.cpu.6.temperature: 43.0C > dev.cpu.6.cx_supported: C1/1 > dev.cpu.6.cx_lowest: C1 > dev.cpu.6.cx_usage: 100.00% last 19304us > dev.cpu.7.%desc: ACPI CPU > dev.cpu.7.%driver: cpu > dev.cpu.7.%location: handle=\_SB_.P007 > dev.cpu.7.%pnpinfo: _HID=none _UID=0 > dev.cpu.7.%parent: acpi0 > dev.cpu.7.coretemp.delta: 48 > dev.cpu.7.coretemp.resolution: 1 > dev.cpu.7.coretemp.tjmax: 91.0C > dev.cpu.7.coretemp.throttle_log: 0 > dev.cpu.7.temperature: 43.0C > dev.cpu.7.cx_supported: C1/1 > dev.cpu.7.cx_lowest: C1 > dev.cpu.7.cx_usage: 100.00% last 17630us > dev.cpu.8.%desc: ACPI CPU > dev.cpu.8.%driver: cpu > dev.cpu.8.%location: handle=\_SB_.P008 > dev.cpu.8.%pnpinfo: _HID=none _UID=0 > dev.cpu.8.%parent: acpi0 > dev.cpu.8.coretemp.delta: 56 > dev.cpu.8.coretemp.resolution: 1 > dev.cpu.8.coretemp.tjmax: 91.0C > dev.cpu.8.coretemp.throttle_log: 0 > dev.cpu.8.temperature: 35.0C > dev.cpu.8.cx_supported: C1/1 > dev.cpu.8.cx_lowest: C1 > dev.cpu.8.cx_usage: 100.00% last 30858us > dev.cpu.9.%desc: ACPI CPU > dev.cpu.9.%driver: cpu > dev.cpu.9.%location: handle=\_SB_.P009 > dev.cpu.9.%pnpinfo: _HID=none _UID=0 > dev.cpu.9.%parent: acpi0 > dev.cpu.9.coretemp.delta: 56 > dev.cpu.9.coretemp.resolution: 1 > dev.cpu.9.coretemp.tjmax: 91.0C > dev.cpu.9.coretemp.throttle_log: 0 > dev.cpu.9.temperature: 35.0C > dev.cpu.9.cx_supported: C1/1 > dev.cpu.9.cx_lowest: C1 > dev.cpu.9.cx_usage: 100.00% last 14794us > dev.cpu.10.%desc: ACPI CPU > dev.cpu.10.%driver: cpu > dev.cpu.10.%location: handle=\_SB_.P00A > dev.cpu.10.%pnpinfo: _HID=none _UID=0 > dev.cpu.10.%parent: acpi0 > dev.cpu.10.coretemp.delta: 50 > dev.cpu.10.coretemp.resolution: 1 > dev.cpu.10.coretemp.tjmax: 91.0C > dev.cpu.10.coretemp.throttle_log: 0 > dev.cpu.10.temperature: 41.0C > dev.cpu.10.cx_supported: C1/1 > dev.cpu.10.cx_lowest: C1 > dev.cpu.10.cx_usage: 100.00% last 12779us > dev.cpu.11.%desc: ACPI CPU > dev.cpu.11.%driver: cpu > dev.cpu.11.%location: handle=\_SB_.P00B > dev.cpu.11.%pnpinfo: _HID=none _UID=0 > dev.cpu.11.%parent: acpi0 > dev.cpu.11.coretemp.delta: 49 > dev.cpu.11.coretemp.resolution: 1 > dev.cpu.11.coretemp.tjmax: 91.0C > dev.cpu.11.coretemp.throttle_log: 0 > dev.cpu.11.temperature: 42.0C > dev.cpu.11.cx_supported: C1/1 > dev.cpu.11.cx_lowest: C1 > dev.cpu.11.cx_usage: 100.00% last 1628us > There was a change (r232793) a few days ago to make turbo boost work, more info in this thread: http://lists.freebsd.org/pipermail/freebsd-current/2012-March/032434.html I wasn't able to get it to work just by tweaking the rc.conf knobs, but I suspect the reason may be because I have non-standard devd.conf. I worked around it and haven't had time to look for the cause yet. In trying to explain the compile time differences between two systems, one of the first things I'd look at would be differences in the disks. In my experience, IO performance has as big an influence on build time as processor speed and number of cores. I just updated my -current sources and did a fresh buildworld and buildkernel using both -j2 and -j12 and had no problems on a 6-core Xeon. I know "make xdev" fails with -jN but I haven't seen failures on any other targets. There was a checkin recently that added some .ORDER stuff for -jN but it only affected building usr.sbin/acpi. -- Ian