From owner-freebsd-bugs@freebsd.org Fri Mar 5 15:45:51 2021 Return-Path: Delivered-To: freebsd-bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D7C0D55DBE1 for ; Fri, 5 Mar 2021 15:45:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4DsXBl5b9Vz4nD2 for ; Fri, 5 Mar 2021 15:45:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id BDD5355DCD9; Fri, 5 Mar 2021 15:45:51 +0000 (UTC) Delivered-To: bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BD95255DBE0 for ; Fri, 5 Mar 2021 15:45:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DsXBl4sdmz4nD1 for ; Fri, 5 Mar 2021 15:45:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9A17312DFE for ; Fri, 5 Mar 2021 15:45:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 125Fjp1u040703 for ; Fri, 5 Mar 2021 15:45:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 125FjpSS040702 for bugs@FreeBSD.org; Fri, 5 Mar 2021 15:45:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 254040] AMD 5950X hyperthreading strange performance swings Date: Fri, 05 Mar 2021 15:45:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: dennis.noordsij@alumni.helsinki.fi X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Mar 2021 15:45:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D254040 Bug ID: 254040 Summary: AMD 5950X hyperthreading strange performance swings Product: Base System Version: 12.2-RELEASE Hardware: amd64 OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: dennis.noordsij@alumni.helsinki.fi I plan to upgrade our server to a Ryzen 9 5950X system, 16 cores, 3400MHz b= ase frequency, 128GB RAM, and ran into an issue while testing. For reference I will use a very simple command: dd if=3D/dev/zero bs=3D1M count=3D1000 | bzip2 - | wc which, using a Linux rescue system (i.e. nothing else running) consistently= and repeatedly completes in about: 1048576000 bytes (1.0 GB, 1000 MiB) copied, 4.67561 s, 224 MB/s Now for FreeBSD 12.2-RELEASE, on a basic boot not running anything except s= sh: if I _disable_ hyperthreading using machdep.hyperthreading_allowed=3D0, I g= et the following approximate result consistently and repeatedly: dd if=3D/dev/zero bs=3D1M count=3D1000 | bzip2 - | wc 1048576000 bytes transferred in 4.874335 secs (215121876 bytes/sec) Slightly slower than Linux, not sure if this is in how bzip2 is compiled et= c, but nothing that worries me.=20 However, if I _enable_ hyperthreading, i.e. the default I started with, the= n I will get: dd if=3D/dev/zero bs=3D1M count=3D1000 | bzip2 - | wc 1048576000 bytes transferred in 4.887522 secs (214541450 bytes/sec) 1048576000 bytes transferred in 7.507138 secs (139677190 bytes/sec) 1048576000 bytes transferred in 6.227179 secs (168386989 bytes/sec) 1048576000 bytes transferred in 7.590263 secs (138147516 bytes/sec) 1048576000 bytes transferred in 7.421037 secs (141297776 bytes/sec) 1048576000 bytes transferred in 4.922986 secs (212995935 bytes/sec) 1048576000 bytes transferred in 4.945138 secs (212041827 bytes/sec) 1048576000 bytes transferred in 7.671600 secs (136682828 bytes/sec) 1048576000 bytes transferred in 7.673428 secs (136650273 bytes/sec) i.e. very consistently varying results with relatively large differences in commands executed immediately after one another (and no other load whatsoev= er). I'm curious why this is happening. I am not running powerd or touched any of the cpu settings. Booting _without_ hyperthreading: CPU: AMD Ryzen 9 5950X 16-Core Processor (3393.70-MHz K8-class = CPU)=20 Origin=3D"AuthenticAMD" Id=3D0xa20f10 Family=3D0x19 Model=3D0x21 Step= ping=3D0=20=20=20=20=20=20=20 =20 Features=3D0x178bfbff=20 =20 Features2=3D0x7ed8320b=20 AMD Features=3D0x2e500800=20=20= =20=20=20=20=20=20=20=20=20=20=20=20 AMD Features2=3D0x75c237ff>=20 Structured Extended Features=3D0x219c97a9=20 Structured Extended Features2=3D0x40068c= =20=20=20=20=20=20=20=20 Structured Extended Features3=3D0x10=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20 XSAVE Features=3D0xf=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 AMD Extended Feature Extensions ID EBX=3D0x111ef657=20=20=20 SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=3D32768=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 TSC: P-state invariant, performance statistics=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 real memory =3D 137434759168 (131068 MB)=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20 avail memory =3D 133793423360 (127595 MB)=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20 Event timer "LAPIC" quality 600=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20 ACPI APIC Table: =20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20 FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 FreeBSD/SMP: 1 package(s) x 2 cache groups x 8 core(s) x 2 hardware threads= =20=20=20=20=20 FreeBSD/SMP Online: 1 package(s) x 2 cache groups x 8 core(s)=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 # sysctl dev.cpu.0=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.cx_metho= d: C1/hlt C2/io=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.cx_usage_counters: 11922 0=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20 dev.cpu.0.cx_usage: 100.00% 0.00% last 43430us=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.cx_lowest: C1=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.cx_supported: C1/1/1 C2/2/18=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20 dev.cpu.0.freq_levels: 3400/3740 2800/2800 2200/1980=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.freq: 3400=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.%parent: acpi0=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.%pnpinfo: _HID=3DACPI0007 _UID=3D0=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20 dev.cpu.0.%location: handle=3D\_SB_.PLTF.C000=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20 dev.cpu.0.%driver: cpu=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.%desc: ACPI CPU=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20 Booting _with_ hyperthreading: CPU: AMD Ryzen 9 5950X 16-Core Processor (3393.69-MHz K8-class = CPU)=20 Origin=3D"AuthenticAMD" Id=3D0xa20f10 Family=3D0x19 Model=3D0x21 Step= ping=3D0=20=20=20=20=20=20=20 =20 Features=3D0x178bfbff=20 =20 Features2=3D0x7ed8320b=20 AMD Features=3D0x2e500800=20=20= =20=20=20=20=20=20=20=20=20=20=20=20 AMD Features2=3D0x75c237ff>=20 Structured Extended Features=3D0x219c97a9=20 Structured Extended Features2=3D0x40068c= =20=20=20=20=20=20=20=20 Structured Extended Features3=3D0x10=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20 XSAVE Features=3D0xf=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 AMD Extended Feature Extensions ID EBX=3D0x111ef657=20=20=20 SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=3D32768=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 TSC: P-state invariant, performance statistics=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 real memory =3D 137434759168 (131068 MB)=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20 avail memory =3D 133793423360 (127595 MB)=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20 Event timer "LAPIC" quality 600=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20 ACPI APIC Table: =20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20 FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 FreeBSD/SMP: 1 package(s) x 2 cache groups x 8 core(s) x 2 hardware threads= =20=20=20=20=20 # sysctl dev.cpu.0=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.cx_method: C1/hlt C2/io=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20 dev.cpu.0.cx_usage_counters: 3232 0=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20 dev.cpu.0.cx_usage: 100.00% 0.00% last 65311us=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.cx_lowest: C1=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.cx_supported: C1/1/1 C2/2/18=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20 dev.cpu.0.freq_levels: 3400/3740 2800/2800 2200/1980=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.freq: 3400=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.%parent: acpi0=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.%pnpinfo: _HID=3DACPI0007 _UID=3D0=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20 dev.cpu.0.%location: handle=3D\_SB_.PLTF.C000=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20 dev.cpu.0.%driver: cpu=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 dev.cpu.0.%desc: ACPI CPU=20 zenstates.py reports: # ./zenstates.py -l P0 - Enabled - FID =3D 88 - DID =3D 8 - VID =3D 48 - IDD =3D 22( / 1 ) - Ratio =3D 34.00 - vCore =3D 1.10000 P1 - Enabled - FID =3D 8C - DID =3D A - VID =3D 58 - IDD =3D 1C( / 1 ) - Ratio =3D 28.00 - vCore =3D 1.00000 P2 - Enabled - FID =3D 84 - DID =3D C - VID =3D 68 - IDD =3D 16( / 1 ) - Ratio =3D 22.00 - vCore =3D 0.90000 P3 - Disabled P4 - Disabled P5 - Disabled P6 - Disabled P7 - Disabled Core Performance Boost - Enabled C6 State - Package - Disabled C6 State - Core - Enabled FWIW if I disable core performance boost the varying execution times shift = from ~4.8 and ~7.8 to to ~6.5 and ~10.8 seconds respectively, i.e. same behaviour just slower. I was hoping someone could explain why this is happening (note it doesn't happen on Linux), if it is expected, and/or how it can be worked around or fixed, or where the problem would be (p-states?).=20 Happy to test anything. PS - I tried a 13-BETA2 rescue boot which has HT enabled and it behaves exa= ctly the same. --=20 You are receiving this mail because: You are the assignee for the bug.=