From nobody Sat Dec 11 17:59:13 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5178518E7952 for ; Sat, 11 Dec 2021 17:59:21 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JBFs43svjz4jdv for ; Sat, 11 Dec 2021 17:59:20 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-oi1-x234.google.com with SMTP id q25so17721408oiw.0 for ; Sat, 11 Dec 2021 09:59:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=9NBHFUSpJQoPD4GlKo/lZs9Enx5GO3cwH4+RRh/SYqU=; b=ZbK2j8wlm3T357TR5Ey4t9tn0Ulh+olGMHuuvOUHXteAxkuuxRiVIoBZ0LQ/rupXkt LZFQ/nxNVE1zTtCFc3T8nQ0qmBg+qz4p4n9vXGFIpZ2y93Iv0wW7Jxx8fVUyhESGJ67z RLWV732gdUwKBjiA3fPJwaEhBoEOulA9uSrMPnfwaUpWH97jWqVGGU7w9Q3F6E4+kOO0 WFXXuseTowQstA57D7VDqqUl+uNwSYA9wPewHVnEB7PI6D/uKQ6ckOL3NY+bFY8IdsI5 yYTNvJC1cStzLHME7ydCmADi4PMiufc22wHrqOeMbEYnDHJIX0qieY+EjWmSqkg95SHF o1wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=9NBHFUSpJQoPD4GlKo/lZs9Enx5GO3cwH4+RRh/SYqU=; b=q7ycbRe8GmUD1v2jutDB8yPzlf/PajySmzrTb1F0WmDdbVTycjLIKC924+QTMgUhWB pKFV4X9dFUhut0LAaupGkSWfocTKSQDp+xP2JptJxHEgNACKBuLbenc373VkQWCVXVB7 c+ODYsI2hspchC+6uboK7bnFo+NCMojP0zGlxfor7OB22gqOL0WJ3CcNMpmXF/k0AQAv 6b+1++H0KEeZ0Vz9gUCEn5YFQFhmAs4Yaz5PV9EBVi01/DSlTMRv/dZdL7TlUwnMHVgV 6sEqUWeRv5cS4fcKnMxfb1xU6dMKylQ25bvo2DLNCk3KzhloxntxrwFoRNzrTIIKZUur XCSA== X-Gm-Message-State: AOAM531bEswQKdj3yNlAUUntco0OOxx+obKP2/JyzFkrXim6NbHE7xFV 9qLCBCbynJrqN8oJhOkQ3i5wcb85nsFnNgVGU7g= X-Google-Smtp-Source: ABdhPJxh6u+PjKQaKIvDSXut6BMyG1TP6qM2xhouQ+ANHk4dpnRZdavLJw6okfb+x6XU0zHR6kLsakzQJSpf3hEg0yc= X-Received: by 2002:a54:4486:: with SMTP id v6mr19102658oiv.90.1639245554402; Sat, 11 Dec 2021 09:59:14 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:6c8c:0:b0:3e8:92e0:314b with HTTP; Sat, 11 Dec 2021 09:59:13 -0800 (PST) In-Reply-To: References: From: Mateusz Guzik Date: Sat, 11 Dec 2021 18:59:13 +0100 Message-ID: Subject: Re: Benchmarks: FreeBSD 13 vs. NetBSD 9.2 vs. OpenBSD 7 vs. DragonFlyBSD 6 vs. Linux To: Piper H Cc: freebsd-current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4JBFs43svjz4jdv X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=ZbK2j8wl; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2607:f8b0:4864:20::234 as permitted sender) smtp.mailfrom=mjguzik@gmail.com X-Spamd-Result: default: False [-3.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.91)[-0.908]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; NEURAL_HAM_LONG(-0.80)[-0.800]; NEURAL_HAM_SHORT(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::234:from]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N On 12/11/21, Mateusz Guzik wrote: > On 12/11/21, Piper H wrote: >> I read this article from Reddit: >> https://www.phoronix.com/scan.php?page=article&item=bsd-linux-eo2021&num=1 >> >> I am surprised to see that the BSD cluster today has much worse >> performance >> than Linux. >> What do you think of this? >> > > There is a lot to say here. > > One has to own up to Linux likely being a little bit (or even more so) > faster for some of the legitimate tests. One, there are certain > multicore scalability issues compared Linux, which should be pretty > mild given the scale (16 cores/32 threads). A more important problem > is userspace which fails to take advantage of SIMD instructions for > core primitives like memset, memcpy et al. However, if the difference > is more than few %, the result is likely bogus. Key thing to do when > benchmarking is being able to explain the result, most notably if you > run into huge discrepancies. > > I had a look at the most egregious result -- zstd and spoiler, it is a > bug in result reporting in zstd. > > I got FreeBSD and Linux (Ubuntu Focal) vms running on: > Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz > > Their zstd test ultimately ends up spawning: zstd -T24 -S -i15 -b19 > FreeBSD-12.2-RELEASE-amd64-memstick.img (yes, they compress a ~1GB > FreeBSD image). > > Side note, it does not matter, but I happen to have CURRENT kernel > running on the FreeBSD 13 vm right now. > > [16:37] freebsd13:~ # time zstd -T24 -S -i15 -b19 > FreeBSD-12.2-RELEASE-amd64-memstick.img > 19#md64-memstick.img :1055957504 -> 692662162 (1.524), 3.97 MB/s ,2156.8 > MB/s > zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img > 274.10s user 12.90s system 763% cpu 37.602 total > > In contrast: > > [16:37] ubuntu:...tem/compress-zstd-1.5.0 (130) # time zstd -T24 -S > -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img > 19#md64-memstick.img :1055957504 -> 692662162 (1.524), 60.1 MB/s ,2030.6 > MB/s > zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img > 328.65s user 3.48s system 850% cpu 39.070 total > > This is repeatable. If anything, FreeBSD did it *faster*. Yet zstd reports: > FreeBSD: 3.97 MB/s ,2156.8 MB/s [total time real time of 37.602 seconds] > Linux: 60.1 MB/s ,2030.6 MB/s [total time real time of 39.070 seconds] > > I don't know what these numbers are supposed to be, but it is pretty > clear Phoronix grabs the first one. > > I'll look into sorting this out some time later. > So I cloned https://github.com/facebook/zstd/ and got the v1.4.8 tag, as currently imported into FreeBSD. The diff is pretty minimal and deals with exposing extra symbols. zstd directly compiled from that source (with mere gmake) correctly shows 2-digit MB speeds, so it has to be something in the FreeBSD build which ends up messing with it. I ran out of curiosity at this point (and more so time) at this point, but I invite someone else to get to the bottom of this. Bottom line though: there is no zstd performance problem on FreeBSD. -- Mateusz Guzik