Date: Sun, 1 May 2005 12:11:44 +0100 (BST) From: Robert Watson <rwatson@FreeBSD.org> To: Kazuaki Oda <kaakun@highway.ne.jp> Cc: andre@FreeBSD.org Subject: Re: tcp output question Message-ID: <20050501114842.S66519@fledge.watson.org> In-Reply-To: <4267E009.6010102@highway.ne.jp> References: <4267E009.6010102@highway.ne.jp>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 22 Apr 2005, Kazuaki Oda wrote: > I'm reading FreeBSD's network codes, and I have noticed that we call > tcp_output() from tcp_usr_send() with tcbinfo locked. According to the > comment in the tcp_usr_send(), we must call tcp_connect() or > tcp_usrclosed() with tcbinfo locked. But it seems that we does not need > to lock it to call tcp_output(). Is there any reason not to unlock it > before calling tcp_output()? I have tried the attached patch, so I get > about 10-20% performance up when running my test server program. You are correct. While it's less performance critical, the same approach can be applied to the PRUS_OOB branch of tcp_usr_send(). There is currently some ambiguity in the relationship between sockets and pcbs -- when various pointers can be NULL, etc. The details vary by protocol -- TCP can allow pcbs to not have sockets, for example, and some socket calls can be made during detach such that there might not be a pcb. If we resolve this ambiguity, we can lower locking overhead, and contention of the tcbinfo lock, quite a bit, avoiding unnecessary acquisition of the tcbinfo lock. Once I've finished cleaning things up elsewhere, I think we need to look carefully at the locking strategy taken for TCP and UDP and decide if they provide the right degree of granularity and overhead. Right now, the tcbinfo/udbinfo locks are held for large parts of TCP and UDP processing, acting as an implicit reference count to prevent pcbs from being free'd while in use in the in-bound delivery path. This means relatively little possible parallel processing of in-bound packets, and also typically means we hit two locks in the TCP/UDP code. Alternative models need to be explored that can help with this, both reducing contention and overhead. Thanks for the patch! Robert N M Watson
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050501114842.S66519>