Date: Tue, 27 Mar 2018 09:53:11 +0000 From: "Kononov, Oleksandr" <oleksandr.kononov@intel.com> To: "freebsd-drivers@freebsd.org" <freebsd-drivers@freebsd.org> Cc: "Vanco, Juraj" <juraj.vanco@intel.com> Subject: FreeBSD 11.1 contigfree performance issue Message-ID: <865AA1660A1A014C99D99B800FA40800813681@IRSMSX101.ger.corp.intel.com>
next in thread | raw e-mail | index | archive | help
--_005_865AA1660A1A014C99D99B800FA40800813681IRSMSX101gercorpi_ Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable I am using FreeBSD 11.1 -RELEASE-amd64 running on a single 32 core CPU and = am having issues with contigmalloc performance. Timing the function using rdtsc shows that it uses up on average about 10 m= illion cycles on that function along. Using the same code, FreeBSD and timing method I ran it on anther machine o= n two CPU's with a total of 32 cores. This gave about 12 thousand cycles on that function. Digging through the source code (on the single CPU) I found that smp_targeted_tlb_shootdown function in /usr/src/sys/x86/x86/mp_x86.c cause the majority of performance hit due to some cores remaining in a paus= ed state longer after the interrupt was send to them. I attached a sample code and Makefile in this email. Steps to recreate (and show rdtsc cycles): $ make $ kldload ./test.ko $ dmesg If anyone has any idea what is the cause of this issue, it would be greatly= appreciated. -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the s= ole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact = the sender and delete all copies. --_005_865AA1660A1A014C99D99B800FA40800813681IRSMSX101gercorpi_ Content-Type: application/octet-stream; name="Makefile" Content-Description: Makefile Content-Disposition: attachment; filename="Makefile"; size=317; creation-date="Tue, 27 Mar 2018 09:51:01 GMT"; modification-date="Tue, 27 Mar 2018 09:20:17 GMT" Content-Transfer-Encoding: base64 IyBOb3RlOiBJdCBpcyBpbXBvcnRhbnQgdG8gbWFrZSBzdXJlIHlvdSBpbmNsdWRlIHRoZSA8YnNk Lmttb2QubWs+IG1ha2VmaWxlIGFmdGVyIGRlY2xhcmluZyB0aGUgS01PRCBhbmQgU1JDUyB2YXJp YWJsZXMuCgojIERlY2xhcmUgTmFtZSBvZiBrZXJuZWwgbW9kdWxlCktNT0QgPSAgIHRlc3QKIyBF bnVtZXJhdGUgU291cmNlIGZpbGVzIGZvciBrZXJuZWwgbW9kdWxlClNSQ1MgPSAgIHRlc3QuYwoK IyBJbmNsdWRlIGtlcm5lbCBtb2R1bGUgbWFrZWZpbGUKIyAvdXNyL3NyYy9zaGFyZS9tay9ic2Qu a21vZC5tawouaW5jbHVkZSA8YnNkLmttb2QubWs+Cgo= --_005_865AA1660A1A014C99D99B800FA40800813681IRSMSX101gercorpi_ Content-Type: text/plain; name="test.c" Content-Description: test.c Content-Disposition: attachment; filename="test.c"; size=1523; creation-date="Tue, 27 Mar 2018 09:50:54 GMT"; modification-date="Tue, 27 Mar 2018 09:20:26 GMT" Content-Transfer-Encoding: base64 I2luY2x1ZGUgPHN5cy9wYXJhbS5oPgojaW5jbHVkZSA8c3lzL21vZHVsZS5oPgojaW5jbHVkZSA8 c3lzL2tlcm5lbC5oPgojaW5jbHVkZSA8c3lzL3N5c3RtLmg+CiNpbmNsdWRlIDxzeXMvbWFsbG9j Lmg+CgpzdGF0aWMgdm9pZCBydW5UZXN0KHZvaWQpOwpNQUxMT0NfREVDTEFSRShNX0ZPTyk7Ck1B TExPQ19ERUZJTkUoTV9GT08sICJtZm9vIiwgImZvbyBtZW0iKTsKCi8qIFRoZSBmdW5jdGlvbiBj YWxsZWQgYXQgbG9hZC91bmxvYWQuICovCnN0YXRpYyBpbnQgZXZlbnRfaGFuZGxlcihzdHJ1Y3Qg bW9kdWxlICptb2R1bGUsIGludCBldmVudCwgdm9pZCAqYXJnKSB7CiAgICBpbnQgZSA9IDA7IC8q IEVycm9yLCAwIGZvciBub3JtYWwgcmV0dXJuIHN0YXR1cyAqLwoKICAgIHN3aXRjaCAoZXZlbnQp IHsKICAgIGNhc2UgTU9EX0xPQUQ6CglydW5UZXN0KCk7CiAgICAgICAgYnJlYWs7CiAgICBjYXNl IE1PRF9VTkxPQUQ6CglwcmludGYoIkRvbmVcbiIpOwoJdXByaW50ZigiRG9uZVxuIik7CiAgICAg ICAgYnJlYWs7CiAgICBkZWZhdWx0OgogICAgICAgIGUgPSBFT1BOT1RTVVBQOyAvKiBFcnJvciwg T3BlcmF0aW9uIE5vdCBTdXBwb3J0ZWQgKi8KICAgICAgICBicmVhazsKICAgIH0KCiAgICByZXR1 cm4oZSk7Cn0KCnN0YXRpYyB2b2lkIHJ1blRlc3QoKSB7CgoJY29uc3QgaW50IHBhZ2VTaXplID0g NDA5NjsKICAgICAgICB1bnNpZ25lZCBsb25nIHNpemU7CiAgICAgICAgdm9pZCAqcHRyID0gTlVM TDsKCXVuc2lnbmVkIGxvbmcgbG9uZyB0MSwgdDI7CgogICAgICAgIGZvcihzaXplID0gMTsgc2l6 ZSA8PSAocGFnZVNpemUgLTEpOyBzaXplIDw8PSAxKSB7CgoJCXQxID0gcmR0c2MoKTsKICAgICAg ICAgICAgICAgIHB0ciA9IGNvbnRpZ21hbGxvYyhzaXplLCBNX0ZPTywgTV9XQUlUT0ssIDAsIH4w LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBhZ2VTaXplLCAwKTsKCQl0MiA9 IHJkdHNjKCk7CgkJcHJpbnRmKCJjb250aWdtYWxsb2MgY3ljbGVzOiAlbGx1XG4iLCB0Mi10MSk7 CgogICAgICAgICAgICAgICAgaWYocHRyID09IE5VTEwpCiAgICAgICAgICAgICAgICAgICAgICAg IHJldHVybjsKCgkJdDEgPSByZHRzYygpOwogICAgICAgICAgICAgICAgY29udGlnZnJlZShwdHIs IHNpemUsIE1fRk9PKTsKCQl0MiA9IHJkdHNjKCk7CgkJcHJpbnRmKCJjb250aWdmcmVlIGN5Y2xl czogJWxsdVxuXG4iLCB0Mi10MSk7CgkJdXByaW50ZigiTG9vcCBjb21wbGV0ZVxuIik7CiAgICAg ICAgfQp9CgovKiBUaGUgc2Vjb25kIGFyZ3VtZW50IG9mIERFQ0xBUkVfTU9EVUxFLiAqLwpzdGF0 aWMgbW9kdWxlZGF0YV90IHRlc3RfY29uZiA9IHsKICAgICJ0ZXN0IiwgICAgLyogbW9kdWxlIG5h bWUgKi8KICAgICBldmVudF9oYW5kbGVyLCAgLyogZXZlbnQgaGFuZGxlciAqLwogICAgIE5VTEwg ICAgICAgICAgICAvKiBleHRyYSBkYXRhICovCn07CgpERUNMQVJFX01PRFVMRSh0ZXN0LCB0ZXN0 X2NvbmYsIFNJX1NVQl9EUklWRVJTLCBTSV9PUkRFUl9NSURETEUpOwo= --_005_865AA1660A1A014C99D99B800FA40800813681IRSMSX101gercorpi_--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?865AA1660A1A014C99D99B800FA40800813681>