Date: Wed, 20 May 2015 07:57:48 -0700 From: Adrian Chadd <adrian@freebsd.org> To: Julien Charbon <jch@freebsd.org> Cc: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: TCP stack lock contention with short-lived connections Message-ID: <CAJ-VmoksQope1GPT%2Bv7tJ68gpXKXVHhpCfk5DeUf_GurVJXcug@mail.gmail.com> In-Reply-To: <555C8BC9.2080304@freebsd.org> References: <op.w51mxed6ak5tgc@fri2jcharbon-m1.local> <op.w56mamc0ak5tgc@dul1rjacobso-l3.vcorp.ad.vrsn.com> <len481$sfv$2@ger.gmane.org> <537F39DF.1090900@verisign.com> <537FB51D.2060401@verisign.com> <537FBFA4.1010902@FreeBSD.org> <53834368.6070103@verisign.com> <555C8BC9.2080304@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 20 May 2015 at 06:27, Julien Charbon <jch@freebsd.org> wrote: > > Hi, > > On 26/05/14 15:36, Julien Charbon wrote: >> On 23/05/14 23:37, Navdeep Parhar wrote: >>> On 05/23/14 13:52, Julien Charbon wrote: >>>> On 23/05/14 14:06, Julien Charbon wrote: >>>>> On 27/02/14 11:32, Julien Charbon wrote: >>>>>> On 07/11/13 14:55, Julien Charbon wrote: >>>>>>> On Mon, 04 Nov 2013 22:21:04 +0100, Julien Charbon >>>>>>> <jcharbon@verisign.com> wrote: >>>>>>>> I have put technical and how-to-repeat details in below >>>>>>>> PR: >>>>>>>> >>>>>>>> kern/183659: TCP stack lock contention with short-lived >>>>>>>> connections >>>>>>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=183659 >>>>>>>> >>>>>>>> We are currently working on this performance improvement >>>>>>>> effort; it will impact only the TCP locking strategy not >>>>>>>> the TCP stack logic itself. We will share on freebsd-net >>>>>>>> the patches we made for reviewing and improvement >>>>>>>> propositions; anyway this change might also require enough >>>>>>>> eyeballs to avoid tricky race conditions introduction in >>>>>>>> TCP stack. >>>> >>>> [...] >> >> Below, just for your information, more details on context of these >> changes: >> >> o The rough consensus at BSDCan was that there is a shared interest for >> scalability improvement of TCP workloads with potential high rate of >> connections establishment and tear-down. >> >> o Our requirements for this task were: >> - Achieve more than 100k TCP connections per second without dropping a >> single packet in reception >> - Use a strategy that does not require to change all network stacks in >> a row (TCP, UDP, RAW, etc.) >> - Be able to progressively introduce better performance, leveraging >> already in place mutex strategy >> - Keep the TCP stack stable (obviously) > > For people interested about this short-lived TCP connection scalability > effort, you can subscribe to the review of our latest (and biggest so > far) change: > > Decompose TCP INP_INFO lock to increase short-lived connections scalability > https://reviews.freebsd.org/D2599 > > The main goal of this review is ideally to start the rough discussion > before BSDCan (and then discuss details in person at BSDCan), and ease > tests by other people with more exotic configurations (thanks Adrian for > your early tests). This patch still improves the short-lived TCP > connection rate (setup and teardown) from 60k/sec to 150k/sec. Hi! I'm using this in our testing lab at work. I get to around 105k/sec, but I think that's primarily because of other lock contention in the kernel (things doing unnecessary ioctl()s.) -adrian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmoksQope1GPT%2Bv7tJ68gpXKXVHhpCfk5DeUf_GurVJXcug>