Date: Thu, 4 Feb 2016 08:44:09 +0000 From: "sepherosa_gmail.com (Sepherosa Ziehau)" <phabric-noreply@FreeBSD.org> To: freebsd-net@freebsd.org Subject: [Differential] [Request, 117 lines] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: <differential-rev-PHID-DREV-wthfupxgtart42sdknmd-req@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
--b1_77fc80858801b21443ab2a903cf15bcc Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit sepherosa_gmail.com created this revision. sepherosa_gmail.com added reviewers: network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, gallatin, hselasky, np. sepherosa_gmail.com added subscribers: freebsd-net-list, freebsd-virtualization-list. Herald added a reviewer: transport. REVISION SUMMARY It's append_cnt based. Unless the network driver sets these two limits, its an NO-OP. For hn(4): - Set TCP ACK append limit to 1, i.e. aggregate 2 ACKs at most. Aggregate anything more than 2 hurts TCP sending performance in hyperv. This significantly improves the TCP sending performance when the number of concurrent connetion is low (2~8). And greatly stabilize the TCP sending performance in other cases. - Set TCP data segments append limit to 25. Without this limitation, hn(4) could aggregate ~45 TCP data segments for each connection (even at 64 or more connections) before dispatching them to socket code; large aggregation slows down ACK sending and eventually hurts/destabilizes TCP reception performance. This setting stabilizes and improves TCP reception performance for >4 concurrent connections significantly. Make them sysctls so they could be adjusted. REVISION DETAIL https://reviews.freebsd.org/D5185 AFFECTED FILES sys/dev/hyperv/netvsc/hv_net_vsc.h sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c sys/netinet/tcp_lro.c sys/netinet/tcp_lro.h EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, transport, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, gallatin, hselasky, np Cc: freebsd-virtualization-list, freebsd-net-list --b1_77fc80858801b21443ab2a903cf15bcc Content-Type: text/x-patch; charset=utf-8; name="D5185.12995.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5185.12995.patch" ZGlmZiAtLWdpdCBhL3N5cy9uZXRpbmV0L3RjcF9scm8uaCBiL3N5cy9uZXRpbmV0L3RjcF9scm8u aAotLS0gYS9zeXMvbmV0aW5ldC90Y3BfbHJvLmgKKysrIGIvc3lzL25ldGluZXQvdGNwX2xyby5o CkBAIC05MSw2ICs5MSw4IEBACiAJdW5zaWduZWQJbHJvX2NudDsKIAl1bnNpZ25lZAlscm9fbWJ1 Zl9jb3VudDsKIAl1bnNpZ25lZAlscm9fbWJ1Zl9tYXg7CisJdW5zaWduZWQgc2hvcnQJbHJvX2Fj a19hcHBlbmRfbGltOworCXVuc2lnbmVkIHNob3J0CWxyb19kYXRhX2FwcGVuZF9saW07CiAKIAlz dHJ1Y3QgbHJvX2hlYWQJbHJvX2FjdGl2ZTsKIAlzdHJ1Y3QgbHJvX2hlYWQJbHJvX2ZyZWU7CmRp ZmYgLS1naXQgYS9zeXMvbmV0aW5ldC90Y3BfbHJvLmMgYi9zeXMvbmV0aW5ldC90Y3BfbHJvLmMK LS0tIGEvc3lzL25ldGluZXQvdGNwX2xyby5jCisrKyBiL3N5cy9uZXRpbmV0L3RjcF9scm8uYwpA QCAtODgsNiArODgsOCBAQAogCWxjLT5scm9fbWJ1Zl9tYXggPSBscm9fbWJ1ZnM7CiAJbGMtPmxy b19jbnQgPSBscm9fZW50cmllczsKIAlsYy0+aWZwID0gaWZwOworCWxjLT5scm9fYWNrX2FwcGVu ZF9saW0gPSAwOworCWxjLT5scm9fZGF0YV9hcHBlbmRfbGltID0gMDsKIAlTTElTVF9JTklUKCZs Yy0+bHJvX2ZyZWUpOwogCVNMSVNUX0lOSVQoJmxjLT5scm9fYWN0aXZlKTsKIApAQCAtNjQ2LDYg KzY0OCwxNiBAQAogCiAJCWlmICh0Y3BfZGF0YV9sZW4gPT0gMCkgewogCQkJbV9mcmVlbShtKTsK KwkJCS8qCisJCQkgKiBGbHVzaCB0aGlzIExSTyBlbnRyeSwgaWYgdGhpcyBBQ0sgc2hvdWxkCisJ CQkgKiBub3QgYmUgZnVydGhlciBkZWxheWVkLgorCQkJICovCisJCQlpZiAobGMtPmxyb19hY2tf YXBwZW5kX2xpbSAmJgorCQkJICAgIGxlLT5hcHBlbmRfY250ID49IGxjLT5scm9fYWNrX2FwcGVu ZF9saW0pIHsKKwkJCQlTTElTVF9SRU1PVkUoJmxjLT5scm9fYWN0aXZlLCBsZSwgbHJvX2VudHJ5 LAorCQkJCSAgICBuZXh0KTsKKwkJCQl0Y3BfbHJvX2ZsdXNoKGxjLCBsZSk7CisJCQl9CiAJCQly ZXR1cm4gKDApOwogCQl9CiAKQEAgLTY2NCw5ICs2NzYsMTIgQEAKIAogCQkvKgogCQkgKiBJZiBh IHBvc3NpYmxlIG5leHQgZnVsbCBsZW5ndGggcGFja2V0IHdvdWxkIGNhdXNlIGFuCi0JCSAqIG92 ZXJmbG93LCBwcm8tYWN0aXZlbHkgZmx1c2ggbm93LgorCQkgKiBvdmVyZmxvdywgcHJvLWFjdGl2 ZWx5IGZsdXNoIG5vdy4gIEFuZCBpZiB3ZSBhcmUgYXNrZWQKKwkJICogdG8gbGltaXQgdGhlIGRh dGEgYWdncmVnYXRlLCBmbHVzaCB0aGlzIExSTyBlbnRyeSBub3cuCiAJCSAqLwotCQlpZiAobGUt PnBfbGVuID4gKDY1NTM1IC0gbGMtPmlmcC0+aWZfbXR1KSkgeworCQlpZiAobGUtPnBfbGVuID4g KDY1NTM1IC0gbGMtPmlmcC0+aWZfbXR1KSB8fAorCQkgICAgKGxjLT5scm9fZGF0YV9hcHBlbmRf bGltICYmCisJCSAgICAgbGUtPmFwcGVuZF9jbnQgPj0gbGMtPmxyb19kYXRhX2FwcGVuZF9saW0p KSB7CiAJCQlTTElTVF9SRU1PVkUoJmxjLT5scm9fYWN0aXZlLCBsZSwgbHJvX2VudHJ5LCBuZXh0 KTsKIAkJCXRjcF9scm9fZmx1c2gobGMsIGxlKTsKIAkJfSBlbHNlCmRpZmYgLS1naXQgYS9zeXMv ZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNkLmMgYi9zeXMvZGV2L2h5cGVy di9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNkLmMKLS0tIGEvc3lzL2Rldi9oeXBlcnYvbmV0 dnNjL2h2X25ldHZzY19kcnZfZnJlZWJzZC5jCisrKyBiL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9o dl9uZXR2c2NfZHJ2X2ZyZWVic2QuYwpAQCAtMTc2LDE0ICsxNzYsOCBAQAogI2RlZmluZSBITl9D U1VNX0FTU0lTVF9XSU44CShDU1VNX1RDUCkKICNkZWZpbmUgSE5fQ1NVTV9BU1NJU1QJCShDU1VN X0lQIHwgQ1NVTV9VRFAgfCBDU1VNX1RDUCkKIAotLyogWFhYIG1vdmUgdG8gbmV0aW5ldC90Y3Bf bHJvLmggKi8KLSNkZWZpbmUgSE5fTFJPX0hJV0FUX01BWAkJCQk2NTUzNQotI2RlZmluZSBITl9M Uk9fSElXQVRfREVGCQkJCUhOX0xST19ISVdBVF9NQVgKLS8qIFlZWSAyKk1UVSBpcyBhIGJpdCBy b3VnaCwgYnV0IHNob3VsZCBiZSBnb29kIGVub3VnaC4gKi8KLSNkZWZpbmUgSE5fTFJPX0hJV0FU X01UVUxJTShpZnApCQkJKDIgKiAoaWZwKS0+aWZfbXR1KQotI2RlZmluZSBITl9MUk9fSElXQVRf SVNWQUxJRChzYywgaGl3YXQpCQkJXAotICAgICgoaGl3YXQpID49IEhOX0xST19ISVdBVF9NVFVM SU0oKHNjKS0+aG5faWZwKSB8fAlcCi0gICAgIChoaXdhdCkgPD0gSE5fTFJPX0hJV0FUX01BWCkK KyNkZWZpbmUgSE5fTFJPX0FDS19BUFBFTkRfTElNCTEKKyNkZWZpbmUgSE5fTFJPX0RBVEFfQVBQ RU5EX0xJTQkyNQogCiAvKgogICogQmUgYXdhcmUgdGhhdCB0aGlzIHNsZWVwYWJsZSBtdXRleCB3 aWxsIGV4aGliaXQgV0lUTkVTUyBlcnJvcnMgd2hlbgpAQCAtMjUzLDI3ICsyNDcsMTYgQEAKIHN0 YXRpYyB2b2lkIGhuX3N0YXJ0X3R4ZW9mKHN0cnVjdCBpZm5ldCAqaWZwKTsKIHN0YXRpYyBpbnQg aG5faWZtZWRpYV91cGQoc3RydWN0IGlmbmV0ICppZnApOwogc3RhdGljIHZvaWQgaG5faWZtZWRp YV9zdHMoc3RydWN0IGlmbmV0ICppZnAsIHN0cnVjdCBpZm1lZGlhcmVxICppZm1yKTsKLSNpZmRl ZiBITl9MUk9fSElXQVQKLXN0YXRpYyBpbnQgaG5fbHJvX2hpd2F0X3N5c2N0bChTWVNDVExfSEFO RExFUl9BUkdTKTsKLSNlbmRpZgogc3RhdGljIGludCBobl90cnVzdF9oY3N1bV9zeXNjdGwoU1lT Q1RMX0hBTkRMRVJfQVJHUyk7CiBzdGF0aWMgaW50IGhuX3R4X2NoaW1uZXlfc2l6ZV9zeXNjdGwo U1lTQ1RMX0hBTkRMRVJfQVJHUyk7CitzdGF0aWMgaW50IGhuX2xyb19hcHBlbmRfbGltX3N5c2N0 bChTWVNDVExfSEFORExFUl9BUkdTKTsKIHN0YXRpYyBpbnQgaG5fY2hlY2tfaXBsZW4oY29uc3Qg c3RydWN0IG1idWYgKiwgaW50KTsKIHN0YXRpYyBpbnQgaG5fY3JlYXRlX3R4X3Jpbmcoc3RydWN0 IGhuX3NvZnRjICpzYyk7CiBzdGF0aWMgdm9pZCBobl9kZXN0cm95X3R4X3Jpbmcoc3RydWN0IGhu X3NvZnRjICpzYyk7CiBzdGF0aWMgdm9pZCBobl9zdGFydF90YXNrZnVuYyh2b2lkICp4c2MsIGlu dCBwZW5kaW5nKTsKIHN0YXRpYyB2b2lkIGhuX3R4ZW9mX3Rhc2tmdW5jKHZvaWQgKnhzYywgaW50 IHBlbmRpbmcpOwogc3RhdGljIGludCBobl9lbmNhcChzdHJ1Y3QgaG5fc29mdGMgKiwgc3RydWN0 IGhuX3R4ZGVzYyAqLCBzdHJ1Y3QgbWJ1ZiAqKik7CiAKLXN0YXRpYyBfX2lubGluZSB2b2lkCi1o bl9zZXRfbHJvX2hpd2F0KHN0cnVjdCBobl9zb2Z0YyAqc2MsIGludCBoaXdhdCkKLXsKLQlzYy0+ aG5fbHJvX2hpd2F0ID0gaGl3YXQ7Ci0jaWZkZWYgSE5fTFJPX0hJV0FUCi0Jc2MtPmhuX2xyby5s cm9faGl3YXQgPSBzYy0+aG5fbHJvX2hpd2F0OwotI2VuZGlmCi19Ci0KIHN0YXRpYyBpbnQKIGhu X2lmbWVkaWFfdXBkKHN0cnVjdCBpZm5ldCAqaWZwIF9fdW51c2VkKQogewpAQCAtMzU4LDcgKzM0 MSw2IEBACiAJYnplcm8oc2MsIHNpemVvZihobl9zb2Z0Y190KSk7CiAJc2MtPmhuX3VuaXQgPSB1 bml0OwogCXNjLT5obl9kZXYgPSBkZXY7Ci0Jc2MtPmhuX2xyb19oaXdhdCA9IEhOX0xST19ISVdB VF9ERUY7CiAJc2MtPmhuX2RpcmVjdF90eF9zaXplID0gaG5fZGlyZWN0X3R4X3NpemU7CiAJaWYg KGhuX3RydXN0X2hvc3R0Y3ApCiAJCXNjLT5obl90cnVzdF9oY3N1bSB8PSBITl9UUlVTVF9IQ1NV TV9UQ1A7CkBAIC00NDIsOSArNDI0LDggQEAKIAkvKiBEcml2ZXIgcHJpdmF0ZSBMUk8gc2V0dGlu Z3MgKi8KIAlzYy0+aG5fbHJvLmlmcCA9IGlmcDsKICNlbmRpZgotI2lmZGVmIEhOX0xST19ISVdB VAotCXNjLT5obl9scm8ubHJvX2hpd2F0ID0gc2MtPmhuX2xyb19oaXdhdDsKLSNlbmRpZgorCXNj LT5obl9scm8ubHJvX2Fja19hcHBlbmRfbGltID0gSE5fTFJPX0FDS19BUFBFTkRfTElNOworCXNj LT5obl9scm8ubHJvX2RhdGFfYXBwZW5kX2xpbSA9IEhOX0xST19EQVRBX0FQUEVORF9MSU07CiAj ZW5kaWYJLyogSU5FVCB8fCBJTkVUNiAqLwogCiAjaWYgX19GcmVlQlNEX3ZlcnNpb24gPj0gMTEw MDA0NQpAQCAtNDcxLDYgKzQ1MiwxMyBAQAogCSAgICBobl90eF9jaGltbmV5X3NpemUgPCBzYy0+ aG5fdHhfY2hpbW5leV9tYXgpCiAJCXNjLT5obl90eF9jaGltbmV5X3NpemUgPSBobl90eF9jaGlt bmV5X3NpemU7CiAKKwkvKgorCSAqIEFsd2F5cyBzY2hlZHVsZSB0cmFuc21pc3Npb24gaW5zdGVh ZCBvZiB0cnlpbmcKKwkgKiB0byBkbyBkaXJlY3QgdHJhbnNtaXNzaW9uLiAgVGhpcyBvbmUgZ2l2 ZXMgdGhlCisJICogYmVzdCBwZXJmb3JtYW5jZSBzbyBmYXIuCisJICovCisJc2MtPmhuX3NjaGVk X3R4ID0gMTsKKwogCWN0eCA9IGRldmljZV9nZXRfc3lzY3RsX2N0eChkZXYpOwogCWNoaWxkID0g U1lTQ1RMX0NISUxEUkVOKGRldmljZV9nZXRfc3lzY3RsX3RyZWUoZGV2KSk7CiAKQEAgLTQ4MCwx MSArNDY4LDYgQEAKIAkgICAgQ1RMRkxBR19SVywgJnNjLT5obl9scm8ubHJvX2ZsdXNoZWQsIDAs ICJMUk8gZmx1c2hlZCIpOwogCVNZU0NUTF9BRERfVUxPTkcoY3R4LCBjaGlsZCwgT0lEX0FVVE8s ICJscm9fdHJpZWQiLAogCSAgICBDVExGTEFHX1JXLCAmc2MtPmhuX2xyb190cmllZCwgIiMgb2Yg TFJPIHRyaWVzIik7Ci0jaWZkZWYgSE5fTFJPX0hJV0FUCi0JU1lTQ1RMX0FERF9QUk9DKGN0eCwg Y2hpbGQsIE9JRF9BVVRPLCAibHJvX2hpd2F0IiwKLQkgICAgQ1RMVFlQRV9JTlQgfCBDVExGTEFH X1JXLCBzYywgMCwgaG5fbHJvX2hpd2F0X3N5c2N0bCwKLQkgICAgIkkiLCAiTFJPIGhpZ2ggd2F0 ZXJtYXJrIik7Ci0jZW5kaWYKIAlTWVNDVExfQUREX1BST0MoY3R4LCBjaGlsZCwgT0lEX0FVVE8s ICJ0cnVzdF9ob3N0dGNwIiwKIAkgICAgQ1RMVFlQRV9JTlQgfCBDVExGTEFHX1JXLCBzYywgSE5f VFJVU1RfSENTVU1fVENQLAogCSAgICBobl90cnVzdF9oY3N1bV9zeXNjdGwsICJJIiwKQEAgLTUz OCw2ICs1MjEsMTQgQEAKIAkgICAgQ1RMRkxBR19SVywgJnNjLT5obl9zY2hlZF90eCwgMCwKIAkg ICAgIkFsd2F5cyBzY2hlZHVsZSB0cmFuc21pc3Npb24gIgogCSAgICAiaW5zdGVhZCBvZiBkb2lu ZyBkaXJlY3QgdHJhbnNtaXNzaW9uIik7CisJU1lTQ1RMX0FERF9QUk9DKGN0eCwgY2hpbGQsIE9J RF9BVVRPLCAibHJvX2Fja19hcHBlbmRfbGltIiwKKwkgICAgQ1RMVFlQRV9JTlQgfCBDVExGTEFH X1JXLCAmc2MtPmhuX2xyby5scm9fYWNrX2FwcGVuZF9saW0sCisJICAgIDAsIGhuX2xyb19hcHBl bmRfbGltX3N5c2N0bCwKKwkgICAgIkkiLCAiTFJPIEFDSyBhcHBlbmRpbmcgbGltaXRhdGlvbiIp OworCVNZU0NUTF9BRERfUFJPQyhjdHgsIGNoaWxkLCBPSURfQVVUTywgImxyb19kYXRhX2FwcGVu ZF9saW0iLAorCSAgICBDVExUWVBFX0lOVCB8IENUTEZMQUdfUlcsICZzYy0+aG5fbHJvLmxyb19k YXRhX2FwcGVuZF9saW0sCisJICAgIDAsIGhuX2xyb19hcHBlbmRfbGltX3N5c2N0bCwKKwkgICAg IkkiLCAiTFJPIGRhdGEgc2VnbWVudHMgYXBwZW5kaW5nIGxpbWl0YXRpb24iKTsKIAogCWlmICh1 bml0ID09IDApIHsKIAkJc3RydWN0IHN5c2N0bF9jdHhfbGlzdCAqZGNfY3R4OwpAQCAtMTQxMCwx MyArMTQwMSw2IEBACiAKIAkJLyogT2J0YWluIGFuZCByZWNvcmQgcmVxdWVzdGVkIE1UVSAqLwog CQlpZnAtPmlmX210dSA9IGlmci0+aWZyX210dTsKLQkJLyoKLQkJICogTWFrZSBzdXJlIHRoYXQg TFJPIGhpZ2ggd2F0ZXJtYXJrIGlzIHN0aWxsIHZhbGlkLAotCQkgKiBhZnRlciBNVFUgY2hhbmdl ICh0aGUgMipNVFUgbGltaXQpLgotCQkgKi8KLQkJaWYgKCFITl9MUk9fSElXQVRfSVNWQUxJRChz Yywgc2MtPmhuX2xyb19oaXdhdCkpCi0JCQlobl9zZXRfbHJvX2hpd2F0KHNjLCBITl9MUk9fSElX QVRfTVRVTElNKGlmcCkpOwotCiAJCWRvIHsKIAkJCU5WX0xPQ0soc2MpOwogCQkJaWYgKCFzYy0+ dGVtcF91bnVzYWJsZSkgewpAQCAtMTcyMiwyNyArMTcwNiw2IEBACiB9CiAjZW5kaWYKIAotI2lm ZGVmIEhOX0xST19ISVdBVAotc3RhdGljIGludAotaG5fbHJvX2hpd2F0X3N5c2N0bChTWVNDVExf SEFORExFUl9BUkdTKQotewotCXN0cnVjdCBobl9zb2Z0YyAqc2MgPSBhcmcxOwotCWludCBoaXdh dCwgZXJyb3I7Ci0KLQloaXdhdCA9IHNjLT5obl9scm9faGl3YXQ7Ci0JZXJyb3IgPSBzeXNjdGxf aGFuZGxlX2ludChvaWRwLCAmaGl3YXQsIDAsIHJlcSk7Ci0JaWYgKGVycm9yIHx8IHJlcS0+bmV3 cHRyID09IE5VTEwpCi0JCXJldHVybiBlcnJvcjsKLQotCWlmICghSE5fTFJPX0hJV0FUX0lTVkFM SUQoc2MsIGhpd2F0KSkKLQkJcmV0dXJuIEVJTlZBTDsKLQotCWlmIChzYy0+aG5fbHJvX2hpd2F0 ICE9IGhpd2F0KQotCQlobl9zZXRfbHJvX2hpd2F0KHNjLCBoaXdhdCk7Ci0JcmV0dXJuIDA7Ci19 Ci0jZW5kaWYJLyogSE5fTFJPX0hJV0FUICovCi0KIHN0YXRpYyBpbnQKIGhuX3RydXN0X2hjc3Vt X3N5c2N0bChTWVNDVExfSEFORExFUl9BUkdTKQogewpAQCAtMTc4Nyw2ICsxNzUwLDI0IEBACiB9 CiAKIHN0YXRpYyBpbnQKK2huX2xyb19hcHBlbmRfbGltX3N5c2N0bChTWVNDVExfSEFORExFUl9B UkdTKQoreworCXVuc2lnbmVkIHNob3J0ICpscm9fbGltID0gYXJnMTsKKwlpbnQgbGltLCBlcnJv cjsKKworCWxpbSA9ICpscm9fbGltOworCWVycm9yID0gc3lzY3RsX2hhbmRsZV9pbnQob2lkcCwg JmxpbSwgMCwgcmVxKTsKKwlpZiAoZXJyb3IgfHwgcmVxLT5uZXdwdHIgPT0gTlVMTCkKKwkJcmV0 dXJuIGVycm9yOworCisJaWYgKGxpbSA8IDAgfHwgbGltID4gNjU1MzUpCisJCXJldHVybiBFSU5W QUw7CisKKwkqbHJvX2xpbSA9IGxpbTsKKwlyZXR1cm4gMDsKK30KKworc3RhdGljIGludAogaG5f Y2hlY2tfaXBsZW4oY29uc3Qgc3RydWN0IG1idWYgKm0sIGludCBob2ZmKQogewogCWNvbnN0IHN0 cnVjdCBpcCAqaXA7CmRpZmYgLS1naXQgYS9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0X3Zz Yy5oIGIvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldF92c2MuaAotLS0gYS9zeXMvZGV2L2h5 cGVydi9uZXR2c2MvaHZfbmV0X3ZzYy5oCisrKyBiL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9u ZXRfdnNjLmgKQEAgLTEwMzAsNyArMTAzMCw2IEBACiAJc3RydWN0IHRhc2sJaG5fdHhlb2ZfdGFz azsKIAogCXN0cnVjdCBscm9fY3RybAlobl9scm87Ci0JaW50CQlobl9scm9faGl3YXQ7CiAKIAkv KiBUcnVzdCBjc3VtIHZlcmlmaWNhdGlvbiBvbiBob3N0IHNpZGUgKi8KIAlpbnQJCWhuX3RydXN0 X2hjc3VtOwkvKiBITl9UUlVTVF9IQ1NVTV8gKi8KCg== --b1_77fc80858801b21443ab2a903cf15bcc--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?differential-rev-PHID-DREV-wthfupxgtart42sdknmd-req>