Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 4 Jan 2018 20:46:55 -0600
From:      Jon Brawn <jon@brawn.org>
To:        David Chisnall <theraven@FreeBSD.org>
Cc:        Nathan Whitehorn <nwhitehorn@freebsd.org>, freebsd-current@freebsd.org
Subject:   Re: Programmatically cache line
Message-ID:  <E6012938-FE63-4459-AAE8-3B469CF18611@brawn.org>
In-Reply-To: <71E8D6E7-F833-4B7E-B1F1-AD07A49CAF98@FreeBSD.org>
References:  <CALM2mEmWYz5nyqvxMJwMWoFOXnDTvWFrEug7UUha6xe7Um6ODw@mail.gmail.com> <20171230082812.GL1684@kib.kiev.ua> <CAJ-VmomxGJsn8eOtWoqevdW-spUPgcSGKEc7eR4xuXLP-E1XRA@mail.gmail.com> <08038E36-9679-4286-9083-FCEDD637ADCC@FreeBSD.org> <20180101103655.GF1684@kib.kiev.ua> <CABh_MK=2uvPoNCg7qL14yVuxo_%2BHVSvccLTBAnRAHNzqor--0g@mail.gmail.com> <35d2d373-92f1-499f-f470-e4528b08b937@freebsd.org> <71E8D6E7-F833-4B7E-B1F1-AD07A49CAF98@FreeBSD.org>

next in thread | previous in thread | raw e-mail | index | archive | help

--Apple-Mail=_02CA1613-A45C-4ED5-BEAA-8CE17DA6E9BB
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8



> On Jan 4, 2018, at 4:03 AM, David Chisnall <theraven@FreeBSD.org> =
wrote:
>=20
> On 3 Jan 2018, at 22:12, Nathan Whitehorn <nwhitehorn@freebsd.org> =
wrote:
>>=20
>> On 01/03/18 13:37, Ed Schouten wrote:
>>> 2018-01-01 11:36 GMT+01:00 Konstantin Belousov =
<kostikbel@gmail.com>:
>>>>>>> On x86, the CPUID instruction leaf 0x1 returns the information =
in
>>>>>>> %ebx register.
>>>>>> Hm, weird. Why don't we extend sysctl to include this info?
>>>> For the same reason we do not provide a sysctl to add two integers.
>>> I strongly agree with Kostik on this one. Why add stuff to the =
kernel,
>>> if userspace is already capable of extracting this? Adding that =
stuff
>>> to sysctl has the downside that it will effectively introduce yet
>>> another FreeBSDism, whereas something generic already exists.
>>>=20
>>=20
>> Well, kind of. The userspace version is platform-dependent and not =
always available: for example, on PPC, you can't do this from userland =
and we provide a sysctl machdep.cacheline_size to userland. It would be =
nice to have an MI API.
>=20
> On ARMv8, similarly, sometimes the kernel needs to advertise the wrong =
size.  A few big.LITTLE cores have 64-byte cache lines on one cluster =
and 32-byte on the other.  If you query the size from userspace while =
running on a 64-byte cluster, then issue the zero-cache-line instruction =
while migrated to the 32-byte cluster, you only clear half the size.  =
Linux works around this by trapping and emulating the instruction to =
query the cache size and always reporting the size for the smallest =
cache lines.  ARM tells people not to build systems like this, but it =
doesn=E2=80=99t always stop them.  Trapping and emulating is much slower =
than just providing the information in a shared page, elf aux args =
vector, or even (often) a system call.
>=20
> To give another example, Linux provides a very cheap way for a =
userspace process to enquire which core it=E2=80=99s running on.  Some =
more recent high-performance mallocs use this to have a second-layer =
per-core cache after the per-thread cache for free blocks.  Unlike the =
per-thread cache, the per-core cache does need a lock, but it=E2=80=99s =
very unlikely to be contended (it will only be contended if either a =
thread is migrated in between checking and locking, so acquires the =
wrong CPU=E2=80=99s lock, or if a thread is preempted in the middle of =
middle of the very brief fill operation).  The author of the SuperMalloc =
paper tried doing this with CPUID and found that it was slower by a =
sufficient margin to almost entirely offset the benefits of the extra =
layer of caching. =20
>=20
> Just because userspace can get at the information directly from the =
hardware doesn=E2=80=99t mean that this is the most efficient or best =
way for userspace to get at it.
>=20
> Oh, and some of these things are useful in portable code, so having to =
write some assembly for every target to get information that the kernel =
already knows is wasteful.
>=20
> David

This idea of Arm big.LITTLE systems having cache lines of different =
lengths really, really bothers me - how on earth is the cache coherency =
supposed to work in such a system? I doubt the usual cache coherency =
protocols would work - probably need a really MESSY protocol to deal =
with this config :-)

Jon.=

--Apple-Mail=_02CA1613-A45C-4ED5-BEAA-8CE17DA6E9BB
Content-Disposition: attachment;
	filename=smime.p7s
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Transfer-Encoding: base64

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIILEzCCBSUw
ggQNoAMCAQICECQcc6QQWc3Um5HgAnmjhBYwDQYJKoZIhvcNAQELBQAwgZcxCzAJBgNVBAYTAkdC
MRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoT
EUNPTU9ETyBDQSBMaW1pdGVkMT0wOwYDVQQDEzRDT01PRE8gUlNBIENsaWVudCBBdXRoZW50aWNh
dGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMB4XDTE3MTAyODAwMDAwMFoXDTE4MTAyODIzNTk1OVow
HjEcMBoGCSqGSIb3DQEJARYNam9uQGJyYXduLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
AQoCggEBALEscuT73gkjXEfkaQU3QXOOIDFilHr9RV/FKPk+ZO3wyXpoChqRW+anE+kKBLSCsmoX
6HnhAmcq3j9umj5jIYwpD84m26XbWQK+uo42GZ3cAF12VvO0g/toUvI+nJcxiD39APWowPKQ4Nae
4FN4hLOcwd2zyF3LiJgq4aXXcBQxl2s1JRCb7STFl5qpp73JVbFp1MkABmESyzI6KE0LLH3hHICU
d2m+Omg6L8T+RgsTEKmgTvw1hYD04ms9ttji/viI8LtR3V9p9DDGH0iSCF56kPo4WfsbfGVBs1km
tw8uvB6OVNGiD0q05kR/GI4jGiMLa4UhlCC0VsYfx7ZyGEUCAwEAAaOCAeMwggHfMB8GA1UdIwQY
MBaAFIKvbIz4xf6WYXzoHz0rcUhexIvAMB0GA1UdDgQWBBRYtBFf7BnRYLxKWDc5DiI35q5WVzAO
BgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAgBgNVHSUEGTAXBggrBgEFBQcDBAYLKwYBBAGy
MQEDBQIwEQYJYIZIAYb4QgEBBAQDAgUgMEYGA1UdIAQ/MD0wOwYMKwYBBAGyMQECAQEBMCswKQYI
KwYBBQUHAgEWHWh0dHBzOi8vc2VjdXJlLmNvbW9kby5uZXQvQ1BTMFoGA1UdHwRTMFEwT6BNoEuG
SWh0dHA6Ly9jcmwuY29tb2RvY2EuY29tL0NPTU9ET1JTQUNsaWVudEF1dGhlbnRpY2F0aW9uYW5k
U2VjdXJlRW1haWxDQS5jcmwwgYsGCCsGAQUFBwEBBH8wfTBVBggrBgEFBQcwAoZJaHR0cDovL2Ny
dC5jb21vZG9jYS5jb20vQ09NT0RPUlNBQ2xpZW50QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFp
bENBLmNydDAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuY29tb2RvY2EuY29tMBgGA1UdEQQRMA+B
DWpvbkBicmF3bi5vcmcwDQYJKoZIhvcNAQELBQADggEBAKZgWVdxinnS81TZvPWc8kXjtzxKSBFU
6ZXBkofX+CSRuD+Wmg4vlt6fNIaVWqWDF95qjR3TOwyb+LQJnsMyYhAl9NI6AJTxgfghzKK49MVP
aC0K7V4TnWCiucJsfK+xDqZIevPFPF3mpYz7/Uf8VPbX2uK80/uUoBRroXDLyHv7fTzG8K+bHBh6
l2x2xFB04nxAhRS4yaJvOeV6ckPOHvCgHhncXQ1HoPUvV/M94K3jaURLPvSUm2tgzODJ97QDHDWM
SF7xfItpAM7AVAmN0M0U8sWI/qDykqpoeOc/TrMNeRTEcuphuJASMuN+oP57T+XZFq/lOEEIw1H+
4QZ1mnIwggXmMIIDzqADAgECAhBqm+E4O/8ra58B1dm4p1JWMA0GCSqGSIb3DQEBDAUAMIGFMQsw
CQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3Jk
MRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDErMCkGA1UEAxMiQ09NT0RPIFJTQSBDZXJ0aWZp
Y2F0aW9uIEF1dGhvcml0eTAeFw0xMzAxMTAwMDAwMDBaFw0yODAxMDkyMzU5NTlaMIGXMQswCQYD
VQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRow
GAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE9MDsGA1UEAxM0Q09NT0RPIFJTQSBDbGllbnQgQXV0
aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
AQoCggEBAL6znlesKHZ1QBbHOAOY08YYdiFQ8yV5C0y1oNF9Olg+nKcxLqf2NHbZhGra0D00SOTq
9bus3/mxgUsg/Wh/eXQ0pnp8tZ8XZWAnlyKMpjL+qUByRjXCA6RQyDMqVaVUkbIr5SU0RDX/kSsK
wer3H1pT/HUrBN0X8sKtPTdGX8XAWt/VdMLBrZBlgvnkCos+KQWWCo63OTTqRvaq8aWccm+KOMjT
cE6s2mj6RkalweyDI7X+7U5lNo6jzC8RTXtVV4/Vwdax720YpMPJQaDaElmOupyTf1Qib+cpukNJ
nQmwygjD8m046DQkLnpXNCAGjuJy1F5NATksUsbfJAr7FLUCAwEAAaOCATwwggE4MB8GA1UdIwQY
MBaAFLuvfgI9+qbxPISOre44mOzZMjLUMB0GA1UdDgQWBBSCr2yM+MX+lmF86B89K3FIXsSLwDAO
BgNVHQ8BAf8EBAMCAYYwEgYDVR0TAQH/BAgwBgEB/wIBADARBgNVHSAECjAIMAYGBFUdIAAwTAYD
VR0fBEUwQzBBoD+gPYY7aHR0cDovL2NybC5jb21vZG9jYS5jb20vQ09NT0RPUlNBQ2VydGlmaWNh
dGlvbkF1dGhvcml0eS5jcmwwcQYIKwYBBQUHAQEEZTBjMDsGCCsGAQUFBzAChi9odHRwOi8vY3J0
LmNvbW9kb2NhLmNvbS9DT01PRE9SU0FBZGRUcnVzdENBLmNydDAkBggrBgEFBQcwAYYYaHR0cDov
L29jc3AuY29tb2RvY2EuY29tMA0GCSqGSIb3DQEBDAUAA4ICAQB4XLKBKDRPPO5fVs6fl1bsj6Jr
F/bz9kkIBtTYLzXN30D+03Hj6OxCDBEaIeNmsBhrJmuubvyE7HtoSmR809AgcYboW+rcTNZ/8u/H
v+GTrNI/AhqX2/kiQNxmgUPt/eJPs92Qclj0HnVyy9TnSvGkSDU7I5Px+TbO+88G4zipA2psZaWe
EykgzClZlPz1FjTCkk77ZXp5cQYYexE6zeeN4/0OqqoAloFrjAF4o50YJafX8mnahjp3I2Y2mkjh
k0xQfhNqbzlLWPoT3m7j7U26u7zg6swjOq8hITYc3/np5tM5aVyu6t99p17bTbY7+1RTWBviN9YJ
zK8HxzObXYWBf/L+VGOYNsQDTxAk0Hbvb1j6KjUhg7fO294F29QIhhmiNOr84JHoy+fNLpfvYc/Q
9EtFOI5ISYgOxLk3nD/whbUe9rmEQXLp8MB933Ij474gwwCPUpwv9mj2PMnXoc7mbrS22XUSeTwx
CTP9bcmUdp4jmIoWfhQm7X9w/Zgddg+JZ/YnIHOwsGsaTUgj7fIvxqith7DoJC91WJ8Lce3CVJqb
1XWeKIJ84F7YLXZN0oa7TktYgDdmQVxYkZo1c5noaDKH9Oq9cbm/vOYRUM1cWcef20Wkyk5S/GFy
yPJwG0fR1nRas3DqAf4cXxMiEKcff7PNa4M3RGTqH0pWR8p6EjGCA7cwggOzAgEBMIGsMIGXMQsw
CQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3Jk
MRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE9MDsGA1UEAxM0Q09NT0RPIFJTQSBDbGllbnQg
QXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIQJBxzpBBZzdSbkeACeaOEFjAJBgUr
DgMCGgUAoIIB3zAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xODAx
MDUwMjQ2NTVaMCMGCSqGSIb3DQEJBDEWBBR5wxRVO7kA0kX2OvnGE3W7yByRRTCBvQYJKwYBBAGC
NxAEMYGvMIGsMIGXMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAw
DgYDVQQHEwdTYWxmb3JkMRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE9MDsGA1UEAxM0Q09N
T0RPIFJTQSBDbGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIQJBxzpBBZ
zdSbkeACeaOEFjCBvwYLKoZIhvcNAQkQAgsxga+ggawwgZcxCzAJBgNVBAYTAkdCMRswGQYDVQQI
ExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoTEUNPTU9ETyBD
QSBMaW1pdGVkMT0wOwYDVQQDEzRDT01PRE8gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQg
U2VjdXJlIEVtYWlsIENBAhAkHHOkEFnN1JuR4AJ5o4QWMA0GCSqGSIb3DQEBAQUABIIBAFhOFrWp
h4kOEQR6OD5azqDg1ZM3B8nG2VS6zlKMRBhGQ2UGXRSuy1nzHTQ8Caed+3uX3W8O4s3GCNQSrpJl
Z44OW+DUDp4rKpSqAp2KVY9x9ttGrMcV8Mb+Fl5FjZ39YEPlj9GPkr+sTklSVnJEugS7Ry9iWA/P
K1VFB8kSgbE7cq3p4+RC62Reqhhl4ldAYO/lKBXBEtX0e+rgWSs2Hj+/u7XeIydydNOpdFU5N+9G
ABCb37ALh9n8s0Q7eZ7w3vFWPKTW1YG/rePXe1X0YhYw+ls4494ZrI0FGc6j+Qcsf7b6rs/0GtoI
ZgDCdPOkQrlhT3pcH0f1zl3AgwMP+RMAAAAAAAA=
--Apple-Mail=_02CA1613-A45C-4ED5-BEAA-8CE17DA6E9BB--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E6012938-FE63-4459-AAE8-3B469CF18611>