From owner-freebsd-current@freebsd.org Thu Jan 4 18:38:14 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 794BFEAD0D8 for ; Thu, 4 Jan 2018 18:38:14 +0000 (UTC) (envelope-from gurenchan@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 39F5278D87; Thu, 4 Jan 2018 18:38:14 +0000 (UTC) (envelope-from gurenchan@gmail.com) Received: by mail-it0-x236.google.com with SMTP id r6so3413550itr.3; Thu, 04 Jan 2018 10:38:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=iqk5d4Jtldn8bo/CtFrU8kqZobm1MAbISFgR5+e6EEo=; b=g2bCj7tp/9IVLkQVN5gC4vJRBWlSGZywBth/zvadUt3KnuloMiVA9NNgvuuco7wr31 CndZs7nMmfVxavf36fCUPQM19ax6NE840oK3b/haGrzXcj3FMcH+g8BjGZqinVW7o1m4 1ZW/1aTMwi9qdT5agE34cPgsTs1oCNjnL1/bM0aiGo8OfXoW1MmK5SlxjDepy/QWF3wy l2UScPibz4PP7sZ/F23MNAXoBNohjgt9TOCyh6c1T2YTumpZsPQ9jI4s7hnn/CDGXB4Y s7nmZ5V/S7QBaUU80EPwfd8D/WFHYXsx4LT+QEmiGlZ6gddzjCr+y3m6Gv347kZx605u 4sgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=iqk5d4Jtldn8bo/CtFrU8kqZobm1MAbISFgR5+e6EEo=; b=fwy51nuAYF/5j/14atUzEBpDSppCPQ253gTTB6gSMNfDpn450cpBPDSGPBIDimpX7n nZrXllLmrtV10bc45OUXKuYlkmOUAd+allJ3jBP3IPf7NjKZamWgBd1lCdlq+BvloQSj gEg0auDPyVlDXUNTCEcSJ/gLJETZAXy9v7GrtaxUpJWLrXZhW1V/JumExPz94nOO+8M0 NQpf7RES2a0TXKKLIhsIYKRmtmQFsk65SWTl6jxVoGqMQuUfRfkEV9o1cP7mLBHyvYIg CUs0GecZo0F3pvPCwXlsm61I4C0uq+eZCvitYMy1KntcXsRjka5KN86BKi+/di8krjXj y7Mg== X-Gm-Message-State: AKGB3mKaOtHcAeuh9xAJ0Yv2erz970x+Zgyi4jm02mNiKPAwXXCVPx6R ejwhpIIRcvWddZKzmT7cVNSjXsfK31yD/TmB+K9LS3Mi X-Google-Smtp-Source: ACJfBot4RZN01u2Bv+SCF13d7y4bQ4QG8MA6TC6zT86Ugt14tRepwDuZzcRxyO1BVdsNFAvwnlHqh6Y2LjO8EeKXij8= X-Received: by 10.36.89.135 with SMTP id p129mr502341itb.100.1515091093408; Thu, 04 Jan 2018 10:38:13 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.164.203 with HTTP; Thu, 4 Jan 2018 10:38:12 -0800 (PST) In-Reply-To: <20180104182940.GY1684@kib.kiev.ua> References: <20171230082812.GL1684@kib.kiev.ua> <08038E36-9679-4286-9083-FCEDD637ADCC@FreeBSD.org> <20180101103655.GF1684@kib.kiev.ua> <35d2d373-92f1-499f-f470-e4528b08b937@freebsd.org> <71E8D6E7-F833-4B7E-B1F1-AD07A49CAF98@FreeBSD.org> <20180104182940.GY1684@kib.kiev.ua> From: blubee blubeeme Date: Fri, 5 Jan 2018 02:38:12 +0800 Message-ID: Subject: Re: Programmatically cache line To: Konstantin Belousov Cc: David Chisnall , Nathan Whitehorn , FreeBSD current Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 18:38:14 -0000 On Fri, Jan 5, 2018 at 2:29 AM, Konstantin Belousov wrote: > On Thu, Jan 04, 2018 at 10:03:32AM +0000, David Chisnall wrote: > > On 3 Jan 2018, at 22:12, Nathan Whitehorn > wrote: > > > > > > On 01/03/18 13:37, Ed Schouten wrote: > > >> 2018-01-01 11:36 GMT+01:00 Konstantin Belousov : > > >>>>>> On x86, the CPUID instruction leaf 0x1 returns the information in > > >>>>>> %ebx register. > > >>>>> Hm, weird. Why don't we extend sysctl to include this info? > > >>> For the same reason we do not provide a sysctl to add two integers. > > >> I strongly agree with Kostik on this one. Why add stuff to the kernel, > > >> if userspace is already capable of extracting this? Adding that stuff > > >> to sysctl has the downside that it will effectively introduce yet > > >> another FreeBSDism, whereas something generic already exists. > > >> > > > > > > Well, kind of. The userspace version is platform-dependent and not > always available: for example, on PPC, you can't do this from userland and > we provide a sysctl machdep.cacheline_size to userland. It would be nice to > have an MI API. > > > > On ARMv8, similarly, sometimes the kernel needs to advertise the wrong > size. A few big.LITTLE cores have 64-byte cache lines on one cluster and > 32-byte on the other. If you query the size from userspace while running > on a 64-byte cluster, then issue the zero-cache-line instruction while > migrated to the 32-byte cluster, you only clear half the size. Linux works > around this by trapping and emulating the instruction to query the cache > size and always reporting the size for the smallest cache lines. ARM tells > people not to build systems like this, but it doesn???t always stop them. > Trapping and emulating is much slower than just providing the information > in a shared page, elf aux args vector, or even (often) a system call. > > Of course MD way is the best way to get such information, just because the > meaning of the 'cache line size' exists only in context of the given CPU > (micro)architecture. For instance, on PowerPC and ARM you are often > concerned > with the granularity of the instruction cache flush, but also you might be > concerned with the DMA, and these are different concepts of cache. > > Even on x86, you may care about alignment to avoid false sharing or > about CLFLUSH granularity, and these can be different legitimately. > Which one to report as 'cache line' ? > > And you cannot bail out with the max among all constants, because sometimes > you really need the min size (for CLFLUSH), and sometime max size (for > false sharing). > > > > > To give another example, Linux provides a very cheap way for a userspace > process to enquire which core it???s running on. Some more recent > high-performance mallocs use this to have a second-layer per-core cache > after the per-thread cache for free blocks. Unlike the per-thread cache, > the per-core cache does need a lock, but it???s very unlikely to be > contended (it will only be contended if either a thread is migrated in > between checking and locking, so acquires the wrong CPU???s lock, or if a > thread is preempted in the middle of middle of the very brief fill > operation). The author of the SuperMalloc paper tried doing this with > CPUID and found that it was slower by a sufficient margin to almost > entirely offset the benefits of the extra layer of caching. > > There, RDTSCP is the intended way to get cpu id in userspace, but the use > of this instruction requires some minimal OS support. It should be faster > than CPUID, since it is not fully serializing. We do not support it only > because nobody asked so far. > > > > > Just because userspace can get at the information directly from the > hardware doesn???t mean that this is the most efficient or best way for > userspace to get at it. > > > It depends, but single instruction (!) vs syscall comparision makes this > discussion silly. > > > Oh, and some of these things are useful in portable code, so having to > write some assembly for every target to get information that the kernel > already knows is wasteful. > > > Required work is to provide the definitions of these interfaces, then they > can be implemented in the best way for each architecture. But nobody did > that. > Initially I was "asking" to see if these facilities were available so that I didn't have to go do some hack job that would be brittle or reinvent the wheel. My use case for getting the cache lines was to prevent false sharing. I should've been more clear about that in my initial email but, I didn't know all these other people were interested in this issue as well. since I'd like the cache line sizes to prevent false sharing that's my use case. Does anyone else have any use cases that they would like to propose? Once we have come to some agreement on what features they need, then we can work out the interfaces and get the work done on implementation? > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >