From owner-freebsd-current@freebsd.org Fri Jan 5 02:46:59 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D5B0EA92EA for ; Fri, 5 Jan 2018 02:46:59 +0000 (UTC) (envelope-from jon@brawn.org) Received: from ahs1.r4l.com (ahs1.r4l.com [198.27.81.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2B9FE6F729; Fri, 5 Jan 2018 02:46:58 +0000 (UTC) (envelope-from jon@brawn.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=brawn.org; s=default; h=References:To:Cc:In-Reply-To:Date:Subject:Mime-Version: Content-Type:Message-Id:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=BhqBFt/KTUPgWaRcfaZ0vYFAbN4dQSYZ8n9si4Inx2s=; b=oOcVbJqBi+kBaNiYwXn2HtuVjZ Qts/qJ1DhMHNQEXgBDi4IPG4FR0z2AYtxJGYBqusx1lpUBH9rv3WXq23sbEWMTML633a34D0G5QqN Zk6bldyjgn9NYTLppf+luNDLhXZFRnbyU85t903aRNmIm9NPcPtqqpL412nGSRPr3pOJjheDQdneY HUd3CWZDf+i2BHT8ogir1I8M+uDID2UPu/iFKi8CcfTa5CLXG5bo51Ru0eSNvklt2Sb3P633dZZvQ maXvSKVRUAgfzNGwvbHa9Ykb53/5btgWCusIGKnpfXl4cLArrVEEV9BeYQViPaFh5EijfOTjSwo1/ 2kja0LVQ==; Received: from [136.62.171.86] (port=49797 helo=[192.168.1.120]) by ahs1.r4l.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89_1) (envelope-from ) id 1eXI25-002Po6-AD; Thu, 04 Jan 2018 21:46:57 -0500 From: Jon Brawn Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_02CA1613-A45C-4ED5-BEAA-8CE17DA6E9BB"; protocol="application/pkcs7-signature"; micalg=sha1 Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Programmatically cache line Date: Thu, 4 Jan 2018 20:46:55 -0600 In-Reply-To: <71E8D6E7-F833-4B7E-B1F1-AD07A49CAF98@FreeBSD.org> Cc: Nathan Whitehorn , freebsd-current@freebsd.org To: David Chisnall References: <20171230082812.GL1684@kib.kiev.ua> <08038E36-9679-4286-9083-FCEDD637ADCC@FreeBSD.org> <20180101103655.GF1684@kib.kiev.ua> <35d2d373-92f1-499f-f470-e4528b08b937@freebsd.org> <71E8D6E7-F833-4B7E-B1F1-AD07A49CAF98@FreeBSD.org> X-Mailer: Apple Mail (2.3445.5.20) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - ahs1.r4l.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - brawn.org X-Get-Message-Sender-Via: ahs1.r4l.com: authenticated_id: jon@brawn.org X-Authenticated-Sender: ahs1.r4l.com: jon@brawn.org X-Source: X-Source-Args: X-Source-Dir: X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 02:46:59 -0000 --Apple-Mail=_02CA1613-A45C-4ED5-BEAA-8CE17DA6E9BB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jan 4, 2018, at 4:03 AM, David Chisnall = wrote: >=20 > On 3 Jan 2018, at 22:12, Nathan Whitehorn = wrote: >>=20 >> On 01/03/18 13:37, Ed Schouten wrote: >>> 2018-01-01 11:36 GMT+01:00 Konstantin Belousov = : >>>>>>> On x86, the CPUID instruction leaf 0x1 returns the information = in >>>>>>> %ebx register. >>>>>> Hm, weird. Why don't we extend sysctl to include this info? >>>> For the same reason we do not provide a sysctl to add two integers. >>> I strongly agree with Kostik on this one. Why add stuff to the = kernel, >>> if userspace is already capable of extracting this? Adding that = stuff >>> to sysctl has the downside that it will effectively introduce yet >>> another FreeBSDism, whereas something generic already exists. >>>=20 >>=20 >> Well, kind of. The userspace version is platform-dependent and not = always available: for example, on PPC, you can't do this from userland = and we provide a sysctl machdep.cacheline_size to userland. It would be = nice to have an MI API. >=20 > On ARMv8, similarly, sometimes the kernel needs to advertise the wrong = size. A few big.LITTLE cores have 64-byte cache lines on one cluster = and 32-byte on the other. If you query the size from userspace while = running on a 64-byte cluster, then issue the zero-cache-line instruction = while migrated to the 32-byte cluster, you only clear half the size. = Linux works around this by trapping and emulating the instruction to = query the cache size and always reporting the size for the smallest = cache lines. ARM tells people not to build systems like this, but it = doesn=E2=80=99t always stop them. Trapping and emulating is much slower = than just providing the information in a shared page, elf aux args = vector, or even (often) a system call. >=20 > To give another example, Linux provides a very cheap way for a = userspace process to enquire which core it=E2=80=99s running on. Some = more recent high-performance mallocs use this to have a second-layer = per-core cache after the per-thread cache for free blocks. Unlike the = per-thread cache, the per-core cache does need a lock, but it=E2=80=99s = very unlikely to be contended (it will only be contended if either a = thread is migrated in between checking and locking, so acquires the = wrong CPU=E2=80=99s lock, or if a thread is preempted in the middle of = middle of the very brief fill operation). The author of the SuperMalloc = paper tried doing this with CPUID and found that it was slower by a = sufficient margin to almost entirely offset the benefits of the extra = layer of caching. =20 >=20 > Just because userspace can get at the information directly from the = hardware doesn=E2=80=99t mean that this is the most efficient or best = way for userspace to get at it. >=20 > Oh, and some of these things are useful in portable code, so having to = write some assembly for every target to get information that the kernel = already knows is wasteful. >=20 > David This idea of Arm big.LITTLE systems having cache lines of different = lengths really, really bothers me - how on earth is the cache coherency = supposed to work in such a system? I doubt the usual cache coherency = protocols would work - probably need a really MESSY protocol to deal = with this config :-) Jon.= --Apple-Mail=_02CA1613-A45C-4ED5-BEAA-8CE17DA6E9BB Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIILEzCCBSUw ggQNoAMCAQICECQcc6QQWc3Um5HgAnmjhBYwDQYJKoZIhvcNAQELBQAwgZcxCzAJBgNVBAYTAkdC MRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoT EUNPTU9ETyBDQSBMaW1pdGVkMT0wOwYDVQQDEzRDT01PRE8gUlNBIENsaWVudCBBdXRoZW50aWNh dGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMB4XDTE3MTAyODAwMDAwMFoXDTE4MTAyODIzNTk1OVow HjEcMBoGCSqGSIb3DQEJARYNam9uQGJyYXduLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC AQoCggEBALEscuT73gkjXEfkaQU3QXOOIDFilHr9RV/FKPk+ZO3wyXpoChqRW+anE+kKBLSCsmoX 6HnhAmcq3j9umj5jIYwpD84m26XbWQK+uo42GZ3cAF12VvO0g/toUvI+nJcxiD39APWowPKQ4Nae 4FN4hLOcwd2zyF3LiJgq4aXXcBQxl2s1JRCb7STFl5qpp73JVbFp1MkABmESyzI6KE0LLH3hHICU d2m+Omg6L8T+RgsTEKmgTvw1hYD04ms9ttji/viI8LtR3V9p9DDGH0iSCF56kPo4WfsbfGVBs1km tw8uvB6OVNGiD0q05kR/GI4jGiMLa4UhlCC0VsYfx7ZyGEUCAwEAAaOCAeMwggHfMB8GA1UdIwQY MBaAFIKvbIz4xf6WYXzoHz0rcUhexIvAMB0GA1UdDgQWBBRYtBFf7BnRYLxKWDc5DiI35q5WVzAO BgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAgBgNVHSUEGTAXBggrBgEFBQcDBAYLKwYBBAGy MQEDBQIwEQYJYIZIAYb4QgEBBAQDAgUgMEYGA1UdIAQ/MD0wOwYMKwYBBAGyMQECAQEBMCswKQYI KwYBBQUHAgEWHWh0dHBzOi8vc2VjdXJlLmNvbW9kby5uZXQvQ1BTMFoGA1UdHwRTMFEwT6BNoEuG SWh0dHA6Ly9jcmwuY29tb2RvY2EuY29tL0NPTU9ET1JTQUNsaWVudEF1dGhlbnRpY2F0aW9uYW5k U2VjdXJlRW1haWxDQS5jcmwwgYsGCCsGAQUFBwEBBH8wfTBVBggrBgEFBQcwAoZJaHR0cDovL2Ny dC5jb21vZG9jYS5jb20vQ09NT0RPUlNBQ2xpZW50QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFp bENBLmNydDAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuY29tb2RvY2EuY29tMBgGA1UdEQQRMA+B DWpvbkBicmF3bi5vcmcwDQYJKoZIhvcNAQELBQADggEBAKZgWVdxinnS81TZvPWc8kXjtzxKSBFU 6ZXBkofX+CSRuD+Wmg4vlt6fNIaVWqWDF95qjR3TOwyb+LQJnsMyYhAl9NI6AJTxgfghzKK49MVP aC0K7V4TnWCiucJsfK+xDqZIevPFPF3mpYz7/Uf8VPbX2uK80/uUoBRroXDLyHv7fTzG8K+bHBh6 l2x2xFB04nxAhRS4yaJvOeV6ckPOHvCgHhncXQ1HoPUvV/M94K3jaURLPvSUm2tgzODJ97QDHDWM SF7xfItpAM7AVAmN0M0U8sWI/qDykqpoeOc/TrMNeRTEcuphuJASMuN+oP57T+XZFq/lOEEIw1H+ 4QZ1mnIwggXmMIIDzqADAgECAhBqm+E4O/8ra58B1dm4p1JWMA0GCSqGSIb3DQEBDAUAMIGFMQsw CQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3Jk MRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDErMCkGA1UEAxMiQ09NT0RPIFJTQSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTAeFw0xMzAxMTAwMDAwMDBaFw0yODAxMDkyMzU5NTlaMIGXMQswCQYD VQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRow GAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE9MDsGA1UEAxM0Q09NT0RPIFJTQSBDbGllbnQgQXV0 aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC AQoCggEBAL6znlesKHZ1QBbHOAOY08YYdiFQ8yV5C0y1oNF9Olg+nKcxLqf2NHbZhGra0D00SOTq 9bus3/mxgUsg/Wh/eXQ0pnp8tZ8XZWAnlyKMpjL+qUByRjXCA6RQyDMqVaVUkbIr5SU0RDX/kSsK wer3H1pT/HUrBN0X8sKtPTdGX8XAWt/VdMLBrZBlgvnkCos+KQWWCo63OTTqRvaq8aWccm+KOMjT cE6s2mj6RkalweyDI7X+7U5lNo6jzC8RTXtVV4/Vwdax720YpMPJQaDaElmOupyTf1Qib+cpukNJ nQmwygjD8m046DQkLnpXNCAGjuJy1F5NATksUsbfJAr7FLUCAwEAAaOCATwwggE4MB8GA1UdIwQY MBaAFLuvfgI9+qbxPISOre44mOzZMjLUMB0GA1UdDgQWBBSCr2yM+MX+lmF86B89K3FIXsSLwDAO BgNVHQ8BAf8EBAMCAYYwEgYDVR0TAQH/BAgwBgEB/wIBADARBgNVHSAECjAIMAYGBFUdIAAwTAYD VR0fBEUwQzBBoD+gPYY7aHR0cDovL2NybC5jb21vZG9jYS5jb20vQ09NT0RPUlNBQ2VydGlmaWNh dGlvbkF1dGhvcml0eS5jcmwwcQYIKwYBBQUHAQEEZTBjMDsGCCsGAQUFBzAChi9odHRwOi8vY3J0 LmNvbW9kb2NhLmNvbS9DT01PRE9SU0FBZGRUcnVzdENBLmNydDAkBggrBgEFBQcwAYYYaHR0cDov L29jc3AuY29tb2RvY2EuY29tMA0GCSqGSIb3DQEBDAUAA4ICAQB4XLKBKDRPPO5fVs6fl1bsj6Jr F/bz9kkIBtTYLzXN30D+03Hj6OxCDBEaIeNmsBhrJmuubvyE7HtoSmR809AgcYboW+rcTNZ/8u/H v+GTrNI/AhqX2/kiQNxmgUPt/eJPs92Qclj0HnVyy9TnSvGkSDU7I5Px+TbO+88G4zipA2psZaWe EykgzClZlPz1FjTCkk77ZXp5cQYYexE6zeeN4/0OqqoAloFrjAF4o50YJafX8mnahjp3I2Y2mkjh k0xQfhNqbzlLWPoT3m7j7U26u7zg6swjOq8hITYc3/np5tM5aVyu6t99p17bTbY7+1RTWBviN9YJ zK8HxzObXYWBf/L+VGOYNsQDTxAk0Hbvb1j6KjUhg7fO294F29QIhhmiNOr84JHoy+fNLpfvYc/Q 9EtFOI5ISYgOxLk3nD/whbUe9rmEQXLp8MB933Ij474gwwCPUpwv9mj2PMnXoc7mbrS22XUSeTwx CTP9bcmUdp4jmIoWfhQm7X9w/Zgddg+JZ/YnIHOwsGsaTUgj7fIvxqith7DoJC91WJ8Lce3CVJqb 1XWeKIJ84F7YLXZN0oa7TktYgDdmQVxYkZo1c5noaDKH9Oq9cbm/vOYRUM1cWcef20Wkyk5S/GFy yPJwG0fR1nRas3DqAf4cXxMiEKcff7PNa4M3RGTqH0pWR8p6EjGCA7cwggOzAgEBMIGsMIGXMQsw CQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3Jk MRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE9MDsGA1UEAxM0Q09NT0RPIFJTQSBDbGllbnQg QXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIQJBxzpBBZzdSbkeACeaOEFjAJBgUr DgMCGgUAoIIB3zAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xODAx MDUwMjQ2NTVaMCMGCSqGSIb3DQEJBDEWBBR5wxRVO7kA0kX2OvnGE3W7yByRRTCBvQYJKwYBBAGC NxAEMYGvMIGsMIGXMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAw DgYDVQQHEwdTYWxmb3JkMRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE9MDsGA1UEAxM0Q09N T0RPIFJTQSBDbGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIQJBxzpBBZ zdSbkeACeaOEFjCBvwYLKoZIhvcNAQkQAgsxga+ggawwgZcxCzAJBgNVBAYTAkdCMRswGQYDVQQI ExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoTEUNPTU9ETyBD QSBMaW1pdGVkMT0wOwYDVQQDEzRDT01PRE8gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQg U2VjdXJlIEVtYWlsIENBAhAkHHOkEFnN1JuR4AJ5o4QWMA0GCSqGSIb3DQEBAQUABIIBAFhOFrWp h4kOEQR6OD5azqDg1ZM3B8nG2VS6zlKMRBhGQ2UGXRSuy1nzHTQ8Caed+3uX3W8O4s3GCNQSrpJl Z44OW+DUDp4rKpSqAp2KVY9x9ttGrMcV8Mb+Fl5FjZ39YEPlj9GPkr+sTklSVnJEugS7Ry9iWA/P K1VFB8kSgbE7cq3p4+RC62Reqhhl4ldAYO/lKBXBEtX0e+rgWSs2Hj+/u7XeIydydNOpdFU5N+9G ABCb37ALh9n8s0Q7eZ7w3vFWPKTW1YG/rePXe1X0YhYw+ls4494ZrI0FGc6j+Qcsf7b6rs/0GtoI ZgDCdPOkQrlhT3pcH0f1zl3AgwMP+RMAAAAAAAA= --Apple-Mail=_02CA1613-A45C-4ED5-BEAA-8CE17DA6E9BB--