Date: Thu, 4 Jan 2018 20:46:55 -0600 From: Jon Brawn <jon@brawn.org> To: David Chisnall <theraven@FreeBSD.org> Cc: Nathan Whitehorn <nwhitehorn@freebsd.org>, freebsd-current@freebsd.org Subject: Re: Programmatically cache line Message-ID: <E6012938-FE63-4459-AAE8-3B469CF18611@brawn.org> In-Reply-To: <71E8D6E7-F833-4B7E-B1F1-AD07A49CAF98@FreeBSD.org> References: <CALM2mEmWYz5nyqvxMJwMWoFOXnDTvWFrEug7UUha6xe7Um6ODw@mail.gmail.com> <20171230082812.GL1684@kib.kiev.ua> <CAJ-VmomxGJsn8eOtWoqevdW-spUPgcSGKEc7eR4xuXLP-E1XRA@mail.gmail.com> <08038E36-9679-4286-9083-FCEDD637ADCC@FreeBSD.org> <20180101103655.GF1684@kib.kiev.ua> <CABh_MK=2uvPoNCg7qL14yVuxo_%2BHVSvccLTBAnRAHNzqor--0g@mail.gmail.com> <35d2d373-92f1-499f-f470-e4528b08b937@freebsd.org> <71E8D6E7-F833-4B7E-B1F1-AD07A49CAF98@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_02CA1613-A45C-4ED5-BEAA-8CE17DA6E9BB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jan 4, 2018, at 4:03 AM, David Chisnall <theraven@FreeBSD.org> = wrote: >=20 > On 3 Jan 2018, at 22:12, Nathan Whitehorn <nwhitehorn@freebsd.org> = wrote: >>=20 >> On 01/03/18 13:37, Ed Schouten wrote: >>> 2018-01-01 11:36 GMT+01:00 Konstantin Belousov = <kostikbel@gmail.com>: >>>>>>> On x86, the CPUID instruction leaf 0x1 returns the information = in >>>>>>> %ebx register. >>>>>> Hm, weird. Why don't we extend sysctl to include this info? >>>> For the same reason we do not provide a sysctl to add two integers. >>> I strongly agree with Kostik on this one. Why add stuff to the = kernel, >>> if userspace is already capable of extracting this? Adding that = stuff >>> to sysctl has the downside that it will effectively introduce yet >>> another FreeBSDism, whereas something generic already exists. >>>=20 >>=20 >> Well, kind of. The userspace version is platform-dependent and not = always available: for example, on PPC, you can't do this from userland = and we provide a sysctl machdep.cacheline_size to userland. It would be = nice to have an MI API. >=20 > On ARMv8, similarly, sometimes the kernel needs to advertise the wrong = size. A few big.LITTLE cores have 64-byte cache lines on one cluster = and 32-byte on the other. If you query the size from userspace while = running on a 64-byte cluster, then issue the zero-cache-line instruction = while migrated to the 32-byte cluster, you only clear half the size. = Linux works around this by trapping and emulating the instruction to = query the cache size and always reporting the size for the smallest = cache lines. ARM tells people not to build systems like this, but it = doesn=E2=80=99t always stop them. Trapping and emulating is much slower = than just providing the information in a shared page, elf aux args = vector, or even (often) a system call. >=20 > To give another example, Linux provides a very cheap way for a = userspace process to enquire which core it=E2=80=99s running on. Some = more recent high-performance mallocs use this to have a second-layer = per-core cache after the per-thread cache for free blocks. Unlike the = per-thread cache, the per-core cache does need a lock, but it=E2=80=99s = very unlikely to be contended (it will only be contended if either a = thread is migrated in between checking and locking, so acquires the = wrong CPU=E2=80=99s lock, or if a thread is preempted in the middle of = middle of the very brief fill operation). The author of the SuperMalloc = paper tried doing this with CPUID and found that it was slower by a = sufficient margin to almost entirely offset the benefits of the extra = layer of caching. =20 >=20 > Just because userspace can get at the information directly from the = hardware doesn=E2=80=99t mean that this is the most efficient or best = way for userspace to get at it. >=20 > Oh, and some of these things are useful in portable code, so having to = write some assembly for every target to get information that the kernel = already knows is wasteful. >=20 > David This idea of Arm big.LITTLE systems having cache lines of different = lengths really, really bothers me - how on earth is the cache coherency = supposed to work in such a system? I doubt the usual cache coherency = protocols would work - probably need a really MESSY protocol to deal = with this config :-) Jon.= --Apple-Mail=_02CA1613-A45C-4ED5-BEAA-8CE17DA6E9BB Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIILEzCCBSUw ggQNoAMCAQICECQcc6QQWc3Um5HgAnmjhBYwDQYJKoZIhvcNAQELBQAwgZcxCzAJBgNVBAYTAkdC MRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoT EUNPTU9ETyBDQSBMaW1pdGVkMT0wOwYDVQQDEzRDT01PRE8gUlNBIENsaWVudCBBdXRoZW50aWNh dGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMB4XDTE3MTAyODAwMDAwMFoXDTE4MTAyODIzNTk1OVow HjEcMBoGCSqGSIb3DQEJARYNam9uQGJyYXduLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC AQoCggEBALEscuT73gkjXEfkaQU3QXOOIDFilHr9RV/FKPk+ZO3wyXpoChqRW+anE+kKBLSCsmoX 6HnhAmcq3j9umj5jIYwpD84m26XbWQK+uo42GZ3cAF12VvO0g/toUvI+nJcxiD39APWowPKQ4Nae 4FN4hLOcwd2zyF3LiJgq4aXXcBQxl2s1JRCb7STFl5qpp73JVbFp1MkABmESyzI6KE0LLH3hHICU d2m+Omg6L8T+RgsTEKmgTvw1hYD04ms9ttji/viI8LtR3V9p9DDGH0iSCF56kPo4WfsbfGVBs1km tw8uvB6OVNGiD0q05kR/GI4jGiMLa4UhlCC0VsYfx7ZyGEUCAwEAAaOCAeMwggHfMB8GA1UdIwQY MBaAFIKvbIz4xf6WYXzoHz0rcUhexIvAMB0GA1UdDgQWBBRYtBFf7BnRYLxKWDc5DiI35q5WVzAO BgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAgBgNVHSUEGTAXBggrBgEFBQcDBAYLKwYBBAGy MQEDBQIwEQYJYIZIAYb4QgEBBAQDAgUgMEYGA1UdIAQ/MD0wOwYMKwYBBAGyMQECAQEBMCswKQYI KwYBBQUHAgEWHWh0dHBzOi8vc2VjdXJlLmNvbW9kby5uZXQvQ1BTMFoGA1UdHwRTMFEwT6BNoEuG SWh0dHA6Ly9jcmwuY29tb2RvY2EuY29tL0NPTU9ET1JTQUNsaWVudEF1dGhlbnRpY2F0aW9uYW5k U2VjdXJlRW1haWxDQS5jcmwwgYsGCCsGAQUFBwEBBH8wfTBVBggrBgEFBQcwAoZJaHR0cDovL2Ny dC5jb21vZG9jYS5jb20vQ09NT0RPUlNBQ2xpZW50QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFp bENBLmNydDAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuY29tb2RvY2EuY29tMBgGA1UdEQQRMA+B DWpvbkBicmF3bi5vcmcwDQYJKoZIhvcNAQELBQADggEBAKZgWVdxinnS81TZvPWc8kXjtzxKSBFU 6ZXBkofX+CSRuD+Wmg4vlt6fNIaVWqWDF95qjR3TOwyb+LQJnsMyYhAl9NI6AJTxgfghzKK49MVP aC0K7V4TnWCiucJsfK+xDqZIevPFPF3mpYz7/Uf8VPbX2uK80/uUoBRroXDLyHv7fTzG8K+bHBh6 l2x2xFB04nxAhRS4yaJvOeV6ckPOHvCgHhncXQ1HoPUvV/M94K3jaURLPvSUm2tgzODJ97QDHDWM SF7xfItpAM7AVAmN0M0U8sWI/qDykqpoeOc/TrMNeRTEcuphuJASMuN+oP57T+XZFq/lOEEIw1H+ 4QZ1mnIwggXmMIIDzqADAgECAhBqm+E4O/8ra58B1dm4p1JWMA0GCSqGSIb3DQEBDAUAMIGFMQsw CQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3Jk MRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDErMCkGA1UEAxMiQ09NT0RPIFJTQSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTAeFw0xMzAxMTAwMDAwMDBaFw0yODAxMDkyMzU5NTlaMIGXMQswCQYD VQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRow GAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE9MDsGA1UEAxM0Q09NT0RPIFJTQSBDbGllbnQgQXV0 aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC AQoCggEBAL6znlesKHZ1QBbHOAOY08YYdiFQ8yV5C0y1oNF9Olg+nKcxLqf2NHbZhGra0D00SOTq 9bus3/mxgUsg/Wh/eXQ0pnp8tZ8XZWAnlyKMpjL+qUByRjXCA6RQyDMqVaVUkbIr5SU0RDX/kSsK wer3H1pT/HUrBN0X8sKtPTdGX8XAWt/VdMLBrZBlgvnkCos+KQWWCo63OTTqRvaq8aWccm+KOMjT cE6s2mj6RkalweyDI7X+7U5lNo6jzC8RTXtVV4/Vwdax720YpMPJQaDaElmOupyTf1Qib+cpukNJ nQmwygjD8m046DQkLnpXNCAGjuJy1F5NATksUsbfJAr7FLUCAwEAAaOCATwwggE4MB8GA1UdIwQY MBaAFLuvfgI9+qbxPISOre44mOzZMjLUMB0GA1UdDgQWBBSCr2yM+MX+lmF86B89K3FIXsSLwDAO BgNVHQ8BAf8EBAMCAYYwEgYDVR0TAQH/BAgwBgEB/wIBADARBgNVHSAECjAIMAYGBFUdIAAwTAYD VR0fBEUwQzBBoD+gPYY7aHR0cDovL2NybC5jb21vZG9jYS5jb20vQ09NT0RPUlNBQ2VydGlmaWNh dGlvbkF1dGhvcml0eS5jcmwwcQYIKwYBBQUHAQEEZTBjMDsGCCsGAQUFBzAChi9odHRwOi8vY3J0 LmNvbW9kb2NhLmNvbS9DT01PRE9SU0FBZGRUcnVzdENBLmNydDAkBggrBgEFBQcwAYYYaHR0cDov L29jc3AuY29tb2RvY2EuY29tMA0GCSqGSIb3DQEBDAUAA4ICAQB4XLKBKDRPPO5fVs6fl1bsj6Jr F/bz9kkIBtTYLzXN30D+03Hj6OxCDBEaIeNmsBhrJmuubvyE7HtoSmR809AgcYboW+rcTNZ/8u/H v+GTrNI/AhqX2/kiQNxmgUPt/eJPs92Qclj0HnVyy9TnSvGkSDU7I5Px+TbO+88G4zipA2psZaWe EykgzClZlPz1FjTCkk77ZXp5cQYYexE6zeeN4/0OqqoAloFrjAF4o50YJafX8mnahjp3I2Y2mkjh k0xQfhNqbzlLWPoT3m7j7U26u7zg6swjOq8hITYc3/np5tM5aVyu6t99p17bTbY7+1RTWBviN9YJ zK8HxzObXYWBf/L+VGOYNsQDTxAk0Hbvb1j6KjUhg7fO294F29QIhhmiNOr84JHoy+fNLpfvYc/Q 9EtFOI5ISYgOxLk3nD/whbUe9rmEQXLp8MB933Ij474gwwCPUpwv9mj2PMnXoc7mbrS22XUSeTwx CTP9bcmUdp4jmIoWfhQm7X9w/Zgddg+JZ/YnIHOwsGsaTUgj7fIvxqith7DoJC91WJ8Lce3CVJqb 1XWeKIJ84F7YLXZN0oa7TktYgDdmQVxYkZo1c5noaDKH9Oq9cbm/vOYRUM1cWcef20Wkyk5S/GFy yPJwG0fR1nRas3DqAf4cXxMiEKcff7PNa4M3RGTqH0pWR8p6EjGCA7cwggOzAgEBMIGsMIGXMQsw CQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3Jk MRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE9MDsGA1UEAxM0Q09NT0RPIFJTQSBDbGllbnQg QXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIQJBxzpBBZzdSbkeACeaOEFjAJBgUr DgMCGgUAoIIB3zAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xODAx MDUwMjQ2NTVaMCMGCSqGSIb3DQEJBDEWBBR5wxRVO7kA0kX2OvnGE3W7yByRRTCBvQYJKwYBBAGC NxAEMYGvMIGsMIGXMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAw DgYDVQQHEwdTYWxmb3JkMRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE9MDsGA1UEAxM0Q09N T0RPIFJTQSBDbGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIQJBxzpBBZ zdSbkeACeaOEFjCBvwYLKoZIhvcNAQkQAgsxga+ggawwgZcxCzAJBgNVBAYTAkdCMRswGQYDVQQI ExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoTEUNPTU9ETyBD QSBMaW1pdGVkMT0wOwYDVQQDEzRDT01PRE8gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQg U2VjdXJlIEVtYWlsIENBAhAkHHOkEFnN1JuR4AJ5o4QWMA0GCSqGSIb3DQEBAQUABIIBAFhOFrWp h4kOEQR6OD5azqDg1ZM3B8nG2VS6zlKMRBhGQ2UGXRSuy1nzHTQ8Caed+3uX3W8O4s3GCNQSrpJl Z44OW+DUDp4rKpSqAp2KVY9x9ttGrMcV8Mb+Fl5FjZ39YEPlj9GPkr+sTklSVnJEugS7Ry9iWA/P K1VFB8kSgbE7cq3p4+RC62Reqhhl4ldAYO/lKBXBEtX0e+rgWSs2Hj+/u7XeIydydNOpdFU5N+9G ABCb37ALh9n8s0Q7eZ7w3vFWPKTW1YG/rePXe1X0YhYw+ls4494ZrI0FGc6j+Qcsf7b6rs/0GtoI ZgDCdPOkQrlhT3pcH0f1zl3AgwMP+RMAAAAAAAA= --Apple-Mail=_02CA1613-A45C-4ED5-BEAA-8CE17DA6E9BB--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E6012938-FE63-4459-AAE8-3B469CF18611>