Date: Tue, 18 Jun 2024 15:08:38 -0600 From: Warner Losh <imp@bsdimp.com> To: Yonas Yanfa <yonas.yanfa@gmail.com> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Comments on the latest phoronix benchmark Message-ID: <CANCZdfp1GFiB0oL8P6qaXS7MWZAMuJv1i_OvOTf7dC2xoi6XKg@mail.gmail.com> In-Reply-To: <93872819-faab-461c-ab13-5a3814edb0d6@gmail.com> References: <93872819-faab-461c-ab13-5a3814edb0d6@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000f854cb061b307c33 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Jun 18, 2024, 2:08=E2=80=AFPM Yonas Yanfa <yonas.yanfa@gmail.com> w= rote: > Hi, > > Link: https://www.phoronix.com/review/bsd-linux-threadripper-7980x/3 > > The following six benchmarks show FreeBSD 14.1 performed poorly compared > to other OSes: > > [ 4.3x slower ] Stress-NG 0.17.08: Socket Activity -- 3,569 (FreeBSD) vs > 15,267 (CentOS Stream 9) > [ 2.9x slower ] Stress-NG 0.17.08: Glibc Qsort Data Sorting -- 779 > (FreeBSD) vs 2,224 (Ubuntu 24.04 LTS) > [ 2.2x slower ] Stress-NG 0.17.08: AVX-512 VNNI -- 3,626,943 (FreeBSD) > vs 8,253,203 (Ubuntu 24.04 LTS) > [ 1.5x slower ] Stress-NG 0.17.08: CPU Cache -- 2,322,478 (FreeBSD) vs > 3,557,329 (NetBSD) > [ 1.5x slower ] Stress-NG 0.17.08: Fused Multiply-Add -- 63,639,465 > (FreeBSD) vs 96,258,730 (Ubuntu 24.04 LTS) > [ 1.3x slower ] Stress-NG 0.17.08: Semaphores -- 230,741,240 (FreeBSD) > vs 313,648,228 (DragonFlyBSD) > > Does anyone know why, and how we can improve the numbers? > Stress-ng is not intended to be a benchmark (and says so in its docs) and does all kinds of special things on Linux only. It has a bunch of stubs on systems that didn=E2=80=99t implement something. It's a deeply flawed. I be= lieve this information is in the comments to the article. That said, there are speed improvements we can make to things, like our VM that other benchmarks do show issues with... but first the benchmarks need to actually be apples to apples comparisons. Warner Cheers, > Yonas > > > --000000000000f854cb061b307c33 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" = class=3D"gmail_attr">On Tue, Jun 18, 2024, 2:08=E2=80=AFPM Yonas Yanfa <= <a href=3D"mailto:yonas.yanfa@gmail.com">yonas.yanfa@gmail.com</a>> wrot= e:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bo= rder-left:1px #ccc solid;padding-left:1ex">Hi,<br> <br> Link: <a href=3D"https://www.phoronix.com/review/bsd-linux-threadripper-798= 0x/3" rel=3D"noreferrer noreferrer" target=3D"_blank">https://www.phoronix.= com/review/bsd-linux-threadripper-7980x/3</a><br> <br> The following six benchmarks show FreeBSD 14.1 performed poorly compared <b= r> to other OSes:<br> <br> [ 4.3x slower ] Stress-NG 0.17.08: Socket Activity -- 3,569 (FreeBSD) vs <b= r> 15,267 (CentOS Stream 9)<br> [ 2.9x slower ] Stress-NG 0.17.08: Glibc Qsort Data Sorting -- 779 <br> (FreeBSD) vs 2,224 (Ubuntu 24.04 LTS)<br> [ 2.2x slower ] Stress-NG 0.17.08: AVX-512 VNNI -- 3,626,943 (FreeBSD) <br> vs 8,253,203 (Ubuntu 24.04 LTS)<br> [ 1.5x slower ] Stress-NG 0.17.08: CPU Cache -- 2,322,478 (FreeBSD) vs <br> 3,557,329 (NetBSD)<br> [ 1.5x slower ] Stress-NG 0.17.08: Fused Multiply-Add -- 63,639,465 <br> (FreeBSD) vs 96,258,730 (Ubuntu 24.04 LTS)<br> [ 1.3x slower ] Stress-NG 0.17.08: Semaphores -- 230,741,240 (FreeBSD) <br> vs 313,648,228 (DragonFlyBSD)<br> <br> Does anyone know why, and how we can improve the numbers?<br></blockquote><= /div></div><div dir=3D"auto"><br></div><div dir=3D"auto">Stress-ng is not i= ntended to be a benchmark (and says so in its docs) and does all kinds of s= pecial things on Linux only. It has a bunch of stubs on systems that didn= =E2=80=99t implement something. It's a deeply flawed. I believe this in= formation is in the comments to the article.</div><div dir=3D"auto"><br></d= iv><div dir=3D"auto">That said, there are speed improvements we can make to= things, like our VM that other benchmarks do show issues with... but first= the benchmarks need to actually be apples to apples comparisons.</div><div= dir=3D"auto"><br></div><div dir=3D"auto">Warner</div><div dir=3D"auto"><br= ></div><div dir=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gm= ail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-le= ft:1ex"> Cheers,<br> Yonas<br> <br> <br> </blockquote></div></div></div> --000000000000f854cb061b307c33--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfp1GFiB0oL8P6qaXS7MWZAMuJv1i_OvOTf7dC2xoi6XKg>