From owner-freebsd-net@FreeBSD.ORG Mon Mar 19 19:54:57 2007 Return-Path: X-Original-To: net@freebsd.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7ACB016A401 for ; Mon, 19 Mar 2007 19:54:57 +0000 (UTC) (envelope-from keith.arner@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.188]) by mx1.freebsd.org (Postfix) with ESMTP id 008EC13C4B0 for ; Mon, 19 Mar 2007 19:54:56 +0000 (UTC) (envelope-from keith.arner@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so15092nfc for ; Mon, 19 Mar 2007 12:54:55 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:references:x-google-sender-auth; b=maczwp+mYlo+i3+A7y4R2JL9Q0QedrY5Nt+HhOmPuubg5kpqrnNjFSpruZDJrMCCW6YiPtgRy/cOylvTJlm/k6wXea0L61PXvBFGAOCc/dkZVkRvlczFyyWawmn8EnJPJffVOZvG0mRX0ekiA9UmNr/1oJtDDxW9dLuxxCfq6NQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:references:x-google-sender-auth; b=kmN2dvgZZ7bpcB6BacZ3lrRYP/NGgPyr4y42w6t5NK5YLIRF0AdkmvZjFAw4whnJOUERDLUOtmWetOXXDI4OqcCFa+BciUtVlnKCDNbjgMWiS4+PWjT2KlWXm72MCOljJXUghxkrmNfBkbdqgbMH+A8CjVAPOjAPfqcr5hEO7Hc= Received: by 10.82.167.5 with SMTP id p5mr10720797bue.1174334095496; Mon, 19 Mar 2007 12:54:55 -0700 (PDT) Received: by 10.82.121.1 with HTTP; Mon, 19 Mar 2007 12:54:55 -0700 (PDT) Message-ID: <8e552a500703191254qba4e194y810eb99a9f07bed8@mail.gmail.com> Date: Mon, 19 Mar 2007 15:54:55 -0400 From: "Keith Arner" Sender: keith.arner@gmail.com To: "Robert Watson" In-Reply-To: <20070311073249.R20646@fledge.watson.org> MIME-Version: 1.0 References: <45C0CA5D.5090903@incunabulum.net> <45E6BEE0.2050307@FreeBSD.org> <45E6C22D.7060200@freebsd.org> <45E6D70C.10104@FreeBSD.org> <45EEB086.3050409@FreeBSD.org> <45F03269.7050705@FreeBSD.org> <45F08F1D.5080708@us.fujitsu.com> <20070310035135.B30274@fledge.watson.org> <8e552a500703102157p1845926au65bb3adaf81c01c0@mail.gmail.com> <20070311073249.R20646@fledge.watson.org> X-Google-Sender-Auth: 9702e7818bd07833 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: net@freebsd.org Subject: Re: netisr_direct X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2007 19:54:57 -0000 On 3/11/07, Robert Watson wrote: > > > There are several ways we could start to reduce contention on that lock: > > (3) Move towards greater granularity of locking for the tcbinfo: instead > of > a single mutex, move to more than one locks, so that different > connections > processed simultaneously are likely to involve different locks. For > listen sockets, we would have to have a special case, such as a > single > lock to serialize simultaneous lock acquisitions of multiple chain > locks > (for example). > I've been thinking about this approach, and it does sound like the simplest to implement of the three proposed. However, the special case of the listen socket seems like it would complicate matters. It seems to me, however, that the complication of the listen socket could be simplified if the listen sockets were maintained in a separate pcb list from the rest of the TCP sockets (the fully connected sockets). If the two types of sockets were thus separated, the code would acquire the lock on the bucket in the connect hash, and search the connect hash; if there was a miss, acquire the lock on the listen list and then search the listen list. The lock on the listen list should follow the locks on the connect buckets in the locking order. The connection bucket should be locked first, as delivery of data would be the common case. To create a new connection in the tcp_input path, both the connect bucket and the listen list would need to be locked (connect bucket as a new entry would be added to the list, and listen list as the accept socket would be protected from going away). Keith