Date: Thu, 4 Jan 2018 20:46:55 -0600 From: Jon Brawn <jon@brawn.org> To: David Chisnall <theraven@FreeBSD.org> Cc: Nathan Whitehorn <nwhitehorn@freebsd.org>, freebsd-current@freebsd.org Subject: Re: Programmatically cache line Message-ID: <E6012938-FE63-4459-AAE8-3B469CF18611@brawn.org> In-Reply-To: <71E8D6E7-F833-4B7E-B1F1-AD07A49CAF98@FreeBSD.org> References: <CALM2mEmWYz5nyqvxMJwMWoFOXnDTvWFrEug7UUha6xe7Um6ODw@mail.gmail.com> <20171230082812.GL1684@kib.kiev.ua> <CAJ-VmomxGJsn8eOtWoqevdW-spUPgcSGKEc7eR4xuXLP-E1XRA@mail.gmail.com> <08038E36-9679-4286-9083-FCEDD637ADCC@FreeBSD.org> <20180101103655.GF1684@kib.kiev.ua> <CABh_MK=2uvPoNCg7qL14yVuxo_%2BHVSvccLTBAnRAHNzqor--0g@mail.gmail.com> <35d2d373-92f1-499f-f470-e4528b08b937@freebsd.org> <71E8D6E7-F833-4B7E-B1F1-AD07A49CAF98@FreeBSD.org>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] > On Jan 4, 2018, at 4:03 AM, David Chisnall <theraven@FreeBSD.org> wrote: > > On 3 Jan 2018, at 22:12, Nathan Whitehorn <nwhitehorn@freebsd.org> wrote: >> >> On 01/03/18 13:37, Ed Schouten wrote: >>> 2018-01-01 11:36 GMT+01:00 Konstantin Belousov <kostikbel@gmail.com>: >>>>>>> On x86, the CPUID instruction leaf 0x1 returns the information in >>>>>>> %ebx register. >>>>>> Hm, weird. Why don't we extend sysctl to include this info? >>>> For the same reason we do not provide a sysctl to add two integers. >>> I strongly agree with Kostik on this one. Why add stuff to the kernel, >>> if userspace is already capable of extracting this? Adding that stuff >>> to sysctl has the downside that it will effectively introduce yet >>> another FreeBSDism, whereas something generic already exists. >>> >> >> Well, kind of. The userspace version is platform-dependent and not always available: for example, on PPC, you can't do this from userland and we provide a sysctl machdep.cacheline_size to userland. It would be nice to have an MI API. > > On ARMv8, similarly, sometimes the kernel needs to advertise the wrong size. A few big.LITTLE cores have 64-byte cache lines on one cluster and 32-byte on the other. If you query the size from userspace while running on a 64-byte cluster, then issue the zero-cache-line instruction while migrated to the 32-byte cluster, you only clear half the size. Linux works around this by trapping and emulating the instruction to query the cache size and always reporting the size for the smallest cache lines. ARM tells people not to build systems like this, but it doesn’t always stop them. Trapping and emulating is much slower than just providing the information in a shared page, elf aux args vector, or even (often) a system call. > > To give another example, Linux provides a very cheap way for a userspace process to enquire which core it’s running on. Some more recent high-performance mallocs use this to have a second-layer per-core cache after the per-thread cache for free blocks. Unlike the per-thread cache, the per-core cache does need a lock, but it’s very unlikely to be contended (it will only be contended if either a thread is migrated in between checking and locking, so acquires the wrong CPU’s lock, or if a thread is preempted in the middle of middle of the very brief fill operation). The author of the SuperMalloc paper tried doing this with CPUID and found that it was slower by a sufficient margin to almost entirely offset the benefits of the extra layer of caching. > > Just because userspace can get at the information directly from the hardware doesn’t mean that this is the most efficient or best way for userspace to get at it. > > Oh, and some of these things are useful in portable code, so having to write some assembly for every target to get information that the kernel already knows is wasteful. > > David This idea of Arm big.LITTLE systems having cache lines of different lengths really, really bothers me - how on earth is the cache coherency supposed to work in such a system? I doubt the usual cache coherency protocols would work - probably need a really MESSY protocol to deal with this config :-) Jon. [-- Attachment #2 --] 0 *H 01 0 + 0 *H 0%0 $ sYԛy0 *H 01 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1=0;U4COMODO RSA Client Authentication and Secure Email CA0 171028000000Z 181028235959Z0 1 0 *H jon@brawn.org0"0 *H 0 ,r #\Gi7As 1bzE_(>dzh [ jyg*?n>c!)&ۥY6 ]vVhR>1= ֞Sxݳ]ˈ*p1k5%$ŗUi a2:(M ,} wi:h:/F N5k=Q_i0 H^z8Y|eAY&. TѢJD## k! V ǶrE 00 U #0la| =+qH^ċ0 U X_`JX79"7VW0U 0 U 0 0 U %0+ +10 `HB 0FU ?0=0; +10+0)+ https://secure.comodo.net/CPS0ZU S0Q0OMKIhttp://crl.comodoca.com/COMODORSAClientAuthenticationandSecureEmailCA.crl0+0}0U+0Ihttp://crt.comodoca.com/COMODORSAClientAuthenticationandSecureEmailCA.crt0$+0http://ocsp.comodoca.com0U 0 jon@brawn.org0 *H `YWqyTټE<JHT$?/ޟ4Zj ; 2b%: !̢Oh- ^`l|Hz<]楌GTkp{}< zlvPt|@ɢo9zrC ] G/W=iDK>k` 5H^|i T ňhx?N yra2~{O8AQur00Πj8;+kٸRV0 *H 01 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1+0)U"COMODO RSA Certification Authority0 130110000000Z 280109235959Z01 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1=0;U4COMODO RSA Client Authentication and Secure Email CA0"0 *H 0 W(vu@8v!P%y L}:X>1.4vلj=4HK hyt4z|e`'"2@rF5P3*UT+%4D5+ ZSu+=7F_Zte >) 94Fro8pNhFF#Ne6/M{UWֱmAYT"o)CI m84$.zW4 r^M9,R$ <080 U #0~=<8220 U la| =+qH^ċ0U 0U 0 0U 00U 0LU E0C0A?=;http://crl.comodoca.com/COMODORSACertificationAuthority.crl0q+e0c0;+0/http://crt.comodoca.com/COMODORSAAddTrustCA.crt0$+0http://ocsp.comodoca.com0 *H x\(4O<_VΟV쏢kI/5@qB !fk&kn{hJd| q[Lǿᓬ?"@fCOݐrX urJH5;#68jle) )Y4Nezyq{: kx%iچ:w#f6HLP~jo9KXnM#:!!6 9i\}^M;TSX7 ̯3]Tc6O$voX*5!4.aKE8HIĹ7? Ar}r# R/h<סnuy<1 3mɔv#~&p vg' skMH#/ƨ$/uX qTu(|^-vM҆NKX7fA\X5sh2qP\YǟENRarpGtZp _" k7Dd JVGz10001 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1=0;U4COMODO RSA Client Authentication and Secure Email CA$ sYԛy0 + 0 *H 1 *H 0 *H 1 180105024655Z0# *H 1yU; E:u E0 +71001 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1=0;U4COMODO RSA Client Authentication and Secure Email CA$ sYԛy0 *H 101 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1=0;U4COMODO RSA Client Authentication and Secure Email CA$ sYԛy0 *H XNz8>ZΠՓ7TRDFCe]Y 4< {oeg[+*UqF^E`CяNIRVrDG/bX+UE;rBd^eW@`({Y+6 ?#'rtөtU97F ߰ D;yV<Ձ{Ub0[8Σ,f tBaOz\ G]help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E6012938-FE63-4459-AAE8-3B469CF18611>