From owner-freebsd-virtualization@FreeBSD.ORG Thu Nov 27 16:57:04 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C5A9106564A for ; Thu, 27 Nov 2008 16:57:04 +0000 (UTC) (envelope-from jorgesanchez75@live.com) Received: from bay0-omc2-s33.bay0.hotmail.com (bay0-omc2-s33.bay0.hotmail.com [65.54.246.169]) by mx1.freebsd.org (Postfix) with ESMTP id 3B0D68FC19 for ; Thu, 27 Nov 2008 16:57:04 +0000 (UTC) (envelope-from jorgesanchez75@live.com) Received: from BAY109-W54 ([64.4.19.154]) by bay0-omc2-s33.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.3959); Thu, 27 Nov 2008 08:57:04 -0800 Message-ID: X-Originating-IP: [70.51.174.19] From: Jorge Sanchez To: Date: Thu, 27 Nov 2008 10:57:04 -0600 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 27 Nov 2008 16:57:04.0176 (UTC) FILETIME=[2C704B00:01C950B1] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Question About TCP Reassembly Inside VImages X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Nov 2008 16:57:04 -0000 Hola Jason=2C I also observed a similar phenomenon on my system's vimages. I have several= thousands dropped packets due to "insufficient memory" (the counter you me= ntion in netstat -m for me is also incremented in the net.inet.tcp.reass.ov= erflows read-only sysctl variable) and I routinely have TCP connections dro= pped within vimages because of it. I think that the TCP packet reassembly q= ueue length is essentially zero once options VIMAGE is enabled... which wou= ld explain my problems when trying to contact hosts that are on flaky links= or are situated very far away hop-wise. I think there is something very wrong with the TCP reassembly when options = VIMAGE is enabled. Did you try increasing the net.inet.reass.maxqlen or net= .inet.reass.maxsegments sysctls? I was able to increase maxqlen but maxsegm= ents must be set in loader.conf and this value is not inherited into any vi= mages I create after booting! :-( If you come up with a fix=2C I would appreciate it too since this prevents = me from performing realistic TCP testing within the virtualization framewor= k. Adios! Jorge From: jason.fines@gmail.com To:=20 freebsd-virtualization@freebsd.org Subject: Re: Question About TCP Reassembly Inside VImages Date: Sat=2C 22 Nov 2008 08:52:16 -0500 Thanks Julian=2C I suspect you are correct as nmbclusters is a system wide sysctl variable set at boot time and although V_tcp_reass_maxseg is set per vimage it is th= e result of a constant operation done on nmbclusters (nmbclusters / 16). What I've described is what I suspect is the root of my problem. The manifestation of this problem is that TCP packets passing through my vimage(s) are not reassembled when they are out of order and I get an exceptionally high value reported by netstat -m stating that packets were dropped due to "insufficient memory". Posts I've found on the net point to the reassembly queue length=2C which in the vimages is zero for some reason= . Perhaps this additional information will help clarify my exact problem. Thanks=2C Jason On Sat=2C Nov 22=2C 2008 at 5:12 AM=2C Julian Elischer wrote: > Jason Fines wrote: > >> Hello all=2C >> >> I've got a question about setting the sysctl variable >> net.inet.tcp.reass.maxsegments to a non-zero value inside my vimages. I= 'm >> currently running the FreeBSD 7 with the VIMAGE package available at >> http://imunes.tel.fer.hr/virtnet/vimage_7-20081015.tgz. >> >> My problem is with TCP reassembly support inside of the vimages=2C namel= y >> with >> the tcp.reass.maxsegments sysctl variable. I've tracked down where in t= he >> code the variable is set to line 122 in tcp_reass_init() of >> netinet/tcp_reass.c: "V_tcp_reass_maxseg =3D nmbclusters / 16=3B". The = line >> clearly reads that maxsegments should be set to "nmbclusters /16"=2C in = the >> main OS (not in any vimage) the value is correctly set to 1/16 of what m= y >> nmbclusters sysctl variable is set to. However=2C inside all my vimages >> nmbclusters is set correctly=2C while reass.maxsegments is incorrectly s= et >> to >> zero!!! >> > > V_tcp_reass_maxseg is a macro that hides the fact that > tcp_reass_maxseg is a PER Vimage variable. > > Part of the patch > is to make some sysctls be per-vimage. I do not know exactly > about that one.. I suspect it is actually a read-only > whole-system value=2C and not per vimage. > > > > > >> Is it possible that nmbclusters when read on line 122 of >> netinet/tcp_reass.c >> is zero? Has anyone else experienced this problem? Is TCP reassembly n= ot >> supported/tested inside vimages? >> >> Any help in this area would be greatly appreciated. >> >> Thanks=2C >> Jason >> >> P.S. This technology is phenomenal=2C and thanks to everyone who is invo= lved >> developing it. >> _______________________________________________ >> freebsd-virtualization at freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe=2C send any mail to " >> freebsd-virtualization-unsubscribe at freebsd.org" >> _________________________________________________________________