Date: Fri, 8 Jan 2016 02:29:22 +0000 From: "sepherosa_gmail.com (Sepherosa Ziehau)" <phabric-noreply@FreeBSD.org> To: freebsd-net@freebsd.org Subject: [Differential] [Request, 6 lines] D4825: tcp/lro: Add network driver configurable LRO entry depth Message-ID: <differential-rev-PHID-DREV-ou2jiti5cx3pzqhm5pb2-req@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
--b1_84e2c2d067c6c1743eb8207d957884aa Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit sepherosa_gmail.com created this revision. sepherosa_gmail.com added reviewers: network, adrian, delphij, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, glebius. sepherosa_gmail.com added a subscriber: freebsd-net-list. Herald added a reviewer: transport. REVISION SUMMARY When there is only tiny amount of TCP connections and the host is slow, e.g. in VM, holding too much TCP segments in an LRO entry will cause RX performance degradation. We now allow network drivers to configure how deep one LRO entry should be. https://reviews.freebsd.org/D4824 has a disabled network driver usage example. Reviewed by: Hongjiang Zhang <honzhan microsoft com>, Dexuan Cui <decui microsoft com>, Jun Su <junsu microsoft com> Tested by: me (local), Hongjiang Zhang <honzhan microsoft com> (directly connected 40Ge) Sponsored by: Microsoft OSTC BTW, I think some drivers already put a limit on the # of drivers holding TCP segments, e.g. oce(4), though oce(4) does not use per-LRO entry depth. REVISION DETAIL https://reviews.freebsd.org/D4825 AFFECTED FILES sys/netinet/tcp_lro.c sys/netinet/tcp_lro.h CHANGE DETAILS diff --git a/sys/netinet/tcp_lro.h b/sys/netinet/tcp_lro.h --- a/sys/netinet/tcp_lro.h +++ b/sys/netinet/tcp_lro.h @@ -79,6 +79,7 @@ int lro_flushed; int lro_bad_csum; int lro_cnt; + int lro_hiwat; struct lro_head lro_active; struct lro_head lro_free; diff --git a/sys/netinet/tcp_lro.c b/sys/netinet/tcp_lro.c --- a/sys/netinet/tcp_lro.c +++ b/sys/netinet/tcp_lro.c @@ -77,6 +77,7 @@ lc->lro_queued = 0; lc->lro_flushed = 0; lc->lro_cnt = 0; + lc->lro_hiwat = 65535; SLIST_INIT(&lc->lro_free); SLIST_INIT(&lc->lro_active); @@ -501,7 +502,7 @@ } /* Flush now if appending will result in overflow. */ - if (le->p_len > (65535 - tcp_data_len)) { + if (le->p_len > (lc->lro_hiwat - tcp_data_len)) { SLIST_REMOVE(&lc->lro_active, le, lro_entry, next); tcp_lro_flush(lc, le); break; @@ -559,7 +560,7 @@ * If a possible next full length packet would cause an * overflow, pro-actively flush now. */ - if (le->p_len > (65535 - lc->ifp->if_mtu)) { + if (le->p_len > (lc->lro_hiwat - lc->ifp->if_mtu)) { SLIST_REMOVE(&lc->lro_active, le, lro_entry, next); tcp_lro_flush(lc, le); } else EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, transport, adrian, delphij, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, glebius Cc: freebsd-net-list --b1_84e2c2d067c6c1743eb8207d957884aa Content-Type: text/x-patch; charset=utf-8; name="D4825.12026.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D4825.12026.patch" ZGlmZiAtLWdpdCBhL3N5cy9uZXRpbmV0L3RjcF9scm8uaCBiL3N5cy9uZXRpbmV0L3RjcF9scm8u aAotLS0gYS9zeXMvbmV0aW5ldC90Y3BfbHJvLmgKKysrIGIvc3lzL25ldGluZXQvdGNwX2xyby5o CkBAIC03OSw2ICs3OSw3IEBACiAJaW50CQlscm9fZmx1c2hlZDsKIAlpbnQJCWxyb19iYWRfY3N1 bTsKIAlpbnQJCWxyb19jbnQ7CisJaW50CQlscm9faGl3YXQ7CiAKIAlzdHJ1Y3QgbHJvX2hlYWQJ bHJvX2FjdGl2ZTsKIAlzdHJ1Y3QgbHJvX2hlYWQJbHJvX2ZyZWU7CmRpZmYgLS1naXQgYS9zeXMv bmV0aW5ldC90Y3BfbHJvLmMgYi9zeXMvbmV0aW5ldC90Y3BfbHJvLmMKLS0tIGEvc3lzL25ldGlu ZXQvdGNwX2xyby5jCisrKyBiL3N5cy9uZXRpbmV0L3RjcF9scm8uYwpAQCAtNzcsNiArNzcsNyBA QAogCWxjLT5scm9fcXVldWVkID0gMDsKIAlsYy0+bHJvX2ZsdXNoZWQgPSAwOwogCWxjLT5scm9f Y250ID0gMDsKKwlsYy0+bHJvX2hpd2F0ID0gNjU1MzU7CiAJU0xJU1RfSU5JVCgmbGMtPmxyb19m cmVlKTsKIAlTTElTVF9JTklUKCZsYy0+bHJvX2FjdGl2ZSk7CiAKQEAgLTUwMSw3ICs1MDIsNyBA QAogCQl9CiAKIAkJLyogRmx1c2ggbm93IGlmIGFwcGVuZGluZyB3aWxsIHJlc3VsdCBpbiBvdmVy Zmxvdy4gKi8KLQkJaWYgKGxlLT5wX2xlbiA+ICg2NTUzNSAtIHRjcF9kYXRhX2xlbikpIHsKKwkJ aWYgKGxlLT5wX2xlbiA+IChsYy0+bHJvX2hpd2F0IC0gdGNwX2RhdGFfbGVuKSkgewogCQkJU0xJ U1RfUkVNT1ZFKCZsYy0+bHJvX2FjdGl2ZSwgbGUsIGxyb19lbnRyeSwgbmV4dCk7CiAJCQl0Y3Bf bHJvX2ZsdXNoKGxjLCBsZSk7CiAJCQlicmVhazsKQEAgLTU1OSw3ICs1NjAsNyBAQAogCQkgKiBJ ZiBhIHBvc3NpYmxlIG5leHQgZnVsbCBsZW5ndGggcGFja2V0IHdvdWxkIGNhdXNlIGFuCiAJCSAq IG92ZXJmbG93LCBwcm8tYWN0aXZlbHkgZmx1c2ggbm93LgogCQkgKi8KLQkJaWYgKGxlLT5wX2xl biA+ICg2NTUzNSAtIGxjLT5pZnAtPmlmX210dSkpIHsKKwkJaWYgKGxlLT5wX2xlbiA+IChsYy0+ bHJvX2hpd2F0IC0gbGMtPmlmcC0+aWZfbXR1KSkgewogCQkJU0xJU1RfUkVNT1ZFKCZsYy0+bHJv X2FjdGl2ZSwgbGUsIGxyb19lbnRyeSwgbmV4dCk7CiAJCQl0Y3BfbHJvX2ZsdXNoKGxjLCBsZSk7 CiAJCX0gZWxzZQoK --b1_84e2c2d067c6c1743eb8207d957884aa--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?differential-rev-PHID-DREV-ou2jiti5cx3pzqhm5pb2-req>