From owner-freebsd-hardware@FreeBSD.ORG Tue Sep 4 09:37:27 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F7D9106566C for ; Tue, 4 Sep 2012 09:37:27 +0000 (UTC) (envelope-from ragnar@gatorhole.com) Received: from maple.lonn.org (maple.lonn.org [109.228.153.253]) by mx1.freebsd.org (Postfix) with ESMTP id E0D1D8FC20 for ; Tue, 4 Sep 2012 09:37:26 +0000 (UTC) Received: from [172.22.21.54] (unknown [82.99.15.130]) (Authenticated sender: ragnar@gatorhole.com) by maple.lonn.org (Postfix) with ESMTPSA id E7B2F735C52; Tue, 4 Sep 2012 11:37:19 +0200 (CEST) Message-ID: <5045CBCB.6000103@gatorhole.com> Date: Tue, 04 Sep 2012 11:37:15 +0200 From: Ragnar Lonn User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Peter Jeremy References: <50446681.2080307@gatorhole.com> <20120903210450.GC2654@aspire.rulingia.com> In-Reply-To: <20120903210450.GC2654@aspire.rulingia.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hardware@freebsd.org Subject: Re: Load testing knocks out network X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 09:37:27 -0000 On 09/03/2012 11:04 PM, Peter Jeremy wrote: > On 2012-Sep-03 10:12:49 +0200, Ragnar Lonn wrote: >> transmit/receive buffers. In newer *Linux* kernels, this memory is being >> allocated in an adaptive manner - i.e. the kernel only allocates a small >> amount of memory to each TCP buffer, and then increases it as necessary >> (per connection, depending on transfer speed and network delay to the >> other peer). > FreeBSD does this as well, though I don't recall when this was added. > >> I think I actually discussed this with FreeBSD developers a while ago >> (on this list even?), and they told me the FreeBSD kernel can only >> allocate max 2GB of kernel memory. > This is only true on 32-bit kernels. FreeBSD uses a single address > space so both kernel and userland need to fit into 4GB on 32-bit > systems. On 64-bit systems, KVM is less constrained (it's ~550GB on > my amd64). You can check sysctl's vm.kvm_free and vm.kvm_size for > exact figures. Maybe I misremembered slightly. I found the old discussion I had with people about this on the FreeBSD virtualization mailing list: http://osdir.com/ml/freebsd-virtualization/2009-02/msg00006.html Anyway, 1.6M connections sounds really good (although he only had 4GB of memory, so I guess the exercise was mostly academic - i.e. those connections would not be very useful in a real setting because each would have so little buffer memory). /Ragnar > >> 100K buffer memory. If you have e.g. 1GB available to network buffers, >> it means a max limit of 10k simultaneous connections on a server, >> regardless of how much memory it has. > If you want a system to usefully cope with 10K network connections, > you will probably want to be running amd64 anyway. That said, Rod > Grimes was achieving between 100K and 1M TCP connections to FreeBSD > i386 systems in the 1990's. >