From nobody Tue Nov 22 19:19:39 2022 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NGvGF2FZPz4hN0X for ; Tue, 22 Nov 2022 19:19:49 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smarthost1.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NGvGF0GpHz3vb0; Tue, 22 Nov 2022 19:19:48 +0000 (UTC) (envelope-from mike@sentex.net) Authentication-Results: mx1.freebsd.org; none Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [199.212.134.19]) by smarthost1.sentex.ca (8.16.1/8.16.1) with ESMTPS id 2AMJJdu7051064 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 22 Nov 2022 14:19:39 -0500 (EST) (envelope-from mike@sentex.net) Received: from [IPV6:2607:f3e0:0:4:bd64:76f:25e7:3d7a] ([IPv6:2607:f3e0:0:4:bd64:76f:25e7:3d7a]) by pyroxene2a.sentex.ca (8.16.1/8.15.2) with ESMTPS id 2AMJJcsh089478 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Tue, 22 Nov 2022 14:19:39 -0500 (EST) (envelope-from mike@sentex.net) Message-ID: Date: Tue, 22 Nov 2022 14:19:39 -0500 List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.5.0 Subject: Re: RELENG_13 and min cpu frequency Content-Language: en-US To: Ian Smith , Kevin Oberman Cc: questions@freebsd.org References: <9d17ea30-4b10-2aa3-9d09-017da7423844@sentex.net> From: mike tancsa In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4NGvGF0GpHz3vb0 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 11/22/2022 12:36 AM, Ian Smith wrote: > > OK, some possible progress. I noticed that > > > >  sysctl -a dev.cpufreq.0.freq_driver > > dev.cpufreq.0.freq_driver: hwpstate_intel0 > > > > Looking at the man page > > > > dev.hwpstate_intel.%d.epp > >              Energy/Performance Preference.  Valid values range from > > 0 to 100. > >              Setting this field conveys a hint to the hardware > > regarding a > >              preference towards performance (at value 0), energy > > efficiency > >              (at value 100), or somewhere in between. > > > > it defaults to 50.  I changed the value to 5 > > > > sysctl -w dev.hwpstate_intel.0.epp=5 > > > > > > Looking at the freq value > > > > dev.cpu.0.freq: 4410 > > dev.cpu.0.freq: 4410 > > dev.cpu.0.freq: 4410 > > dev.cpu.0.freq: 4410 > > dev.cpu.0.freq: 4410 > > dev.cpu.0.freq: 4410 > > dev.cpu.0.freq: 4410 > > dev.cpu.0.freq: 4410 > > dev.cpu.0.freq: 4410 > > dev.cpu.0.freq: 4410 > > > > > > Should have a better sense in a couple of days > > > >     ---Mike > > Ah, well. hwpstate_intel was new in 13, I had to read the man online. > > I guess you'll try various values from 0 to 100? > > Maybe this indicates where powerd or successors might move? > > I noticed also that you can turn it off at boot to perhaps re-engage est(4), or choose between the default package-level or per-cpu control, so there's lots of scope for fine-tuning (or disastrous mismanagement :) > > You might find sysutils/stress helpful with tests; loading up N cpus and/or io and/or memory malloc/free, for specified lengths of time. > > It might be useful to record, say, cpu.0.temperature with frequencies? And maybe vm.loadavg, if it's behaving itself these days ... > > Ah, I really haven't the spare time to be this interested :) > On the main firewall where I first noticed the issue, I have set it to 5 (out of 100) for now and it seems to have made a big difference in terms of dropped packets at slow times. Will try dev.hwpstate_intel.0.epp=0 in a few days.  On my test server, which is the same motherboard and bios rev, but a Xeon with HTT cores, I have been playing a bit and even in synthetic loads like buildkernel seem to make a slight difference. Setting dev.hwpstate_intel.0.epp=0 from the default of 50, makes make -j12 buildkernel up to 5% faster.   About 118 seconds vs 125. Not a huge difference, but one thats consistently there at least with 3 tries. On my production box, the temp pattern according to IPMI hasnt really changed that much.  24hrs of temp readings gives a mean of 45.9C since I set the epp=5. Last Friday by comparison for the same 24hr period had a mean of 43.8.  The fan 2146 RPMs vs 2055. So the fans are working a little harder too, but seems to be in its natural variance. On my test box (same MB, same generation CPU but with HTT) I also tried adding  to /boot/loader.conf hint.hwpstate_intel.0.disabled="1" and sure enough dev.cpufreq.0.freq_driver: est0 dev.cpufreq.0.%driver: cpufreq 0(old-firewall)# sysctl -a dev.cpu.0.freq_levels dev.cpu.0.freq_levels: 3401/80000 3400/80000 3200/73495 3000/67226 2800/61202 2700/58269 2500/52572 2300/47109 2100/41853 1900/36813 1700/32645 1500/28007 1400/25765 1200/21437 1000/17302 800/13362 0(old-firewall)# 0(old-firewall)# sysctl -a dev.cpu.0.freq dev.cpu.0.freq: 3401 0(old-firewall)# There are some settings in the BIOS however, that need to be at their defaults for this older interface to come up.  Anyways, progress :) Although I never thought to check, might be nice to have a note to put in UPDATING for going from RELENG_12 to RELENG_13 that there might be a different default power profile made active even though the same hardware is being used. Disabling hwpstate_intel which is set to a balance of performance and power saving, restores the old default setting to what appears to be max performance on this MB and CPU anyways. Or at least using the newer driver with dev.hwpstate_intel.0.epp=0 restores the same/similar performance as before. I can totally see why having a default of 50 is a good idea, but maybe a slight POLA here.     ---Mike