From owner-freebsd-xen@FreeBSD.ORG Mon May 16 11:07:20 2011 Return-Path: Delivered-To: freebsd-xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5341810656E0 for ; Mon, 16 May 2011 11:07:20 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 279968FC20 for ; Mon, 16 May 2011 11:07:20 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p4GB7KbC071417 for ; Mon, 16 May 2011 11:07:20 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p4GB7Jtx071415 for freebsd-xen@FreeBSD.org; Mon, 16 May 2011 11:07:19 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 16 May 2011 11:07:19 GMT Message-Id: <201105161107.p4GB7Jtx071415@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-xen@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-xen@FreeBSD.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 May 2011 11:07:20 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/155468 xen [xen] Xen PV i386 multi-kernel CPU system is not worki o kern/155353 xen [xen] [patch] put "nudging TOD" message under boot_ver o kern/154833 xen [xen]: xen 4.0 - DomU freebsd8.2RC3 i386, XEN kernel. o kern/154473 xen [xen] xen 4.0 - DomU freebsd8.1 i386, XEN kernel. Not o kern/154472 xen [xen] xen 4.0 - DomU freebsd8.1 i386 xen kernel reboot o kern/154428 xen [xen] xn0 network interface and PF - Massive performan o kern/154302 xen [xen] [panic] [patch] xn0: Error 2 parsing device/vif/ o kern/153789 xen [xen] [regression] FreeBSD 8.2-RC1 crashes under Solar o kern/153674 xen [xen] i386/XEN idle thread shows wrong percentages o kern/153672 xen [xen] [panic] i386/XEN panics under heavy fork load o kern/153620 xen [xen] Xen guest system clock drifts in AWS EC2 (FreeBS o kern/153477 xen [xen] XEN pmap code abuses vm page queue lock o kern/153150 xen [xen] xen/ec2: disable checksum offloading on interfac o kern/152228 xen [xen] [panic] Xen/PV panic with machdep.idle_mwait=1 o kern/144629 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143398 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143340 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor f kern/143069 xen [xen] [panic] Xen Kernel Panic - Memory modified after f kern/135667 xen ufs filesystem corruption on XEN DomU system f kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc. f kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i p kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all f i386/124516 xen [xen] FreeBSD-CURRENT Xen Kernel Segfaults when config o kern/118734 xen [xen] FreeBSD 6.3-RC1 and FreeBSD 7.0-BETA 4 fail to b 24 problems total. From owner-freebsd-xen@FreeBSD.ORG Wed May 18 12:06:23 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDEE71065676 for ; Wed, 18 May 2011 12:06:23 +0000 (UTC) (envelope-from laurent.cligny@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 49F458FC1B for ; Wed, 18 May 2011 12:06:22 +0000 (UTC) Received: by ewy1 with SMTP id 1so634881ewy.13 for ; Wed, 18 May 2011 05:06:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type; bh=nTvGng4GfWUc6mSW1xdMJu41MeThNioDZMKbAohmiFE=; b=sE/Jd+5jm2rPEIBg3PMXJRmZJPTcAaPc2CCflBDFTLDXatO5lI8094apAQ6KdGWlp6 tePOofzJiyYoKOfXuo1OIPNAArxwj71Oa3MaTEYwdZHiWAwFCDSzSEJEewJabTOLei6d hySQ+n5MYUFKHN/i5gkJlSPBP9mI8xQtnTtgU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type; b=ofcDHrtKw9gZTp2dBySD8GitAc80wbPqd/pVMCWwP7vimNISQO0z6kcrHFmLOrN+Wu IanEWLoQHfll/gF84MuFMxSsmkjj62SNkxDv8psRbd2Iq6OzlQRpHx5vZBwmu7kizpdh G9TxxN314sKedo99wsGYZTg4YM2ghN5rpZWLs= Received: by 10.14.0.6 with SMTP id 6mr633796eea.78.1305720381866; Wed, 18 May 2011 05:06:21 -0700 (PDT) Received: from [192.168.1.12] (ARennes-555-1-114-211.w90-49.abo.wanadoo.fr [90.49.97.211]) by mx.google.com with ESMTPS id y3sm1113208eeh.19.2011.05.18.05.06.17 (version=SSLv3 cipher=OTHER); Wed, 18 May 2011 05:06:18 -0700 (PDT) Message-ID: <4DD3B63D.60205@gmail.com> Date: Wed, 18 May 2011 14:06:21 +0200 From: "laurent.cligny@gmail.com" User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110402 Iceowl/1.0b2 Icedove/3.1.9 MIME-Version: 1.0 To: Michael MacLeod References: <4DCBEEE0.9060705@steadinet.fr> <4DCBF681.7070106@ntelecom.com.br> <4DCC1097.4080003@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-xen@freebsd.org Subject: Re: [FreeBSD 8.2 amd64 XENHVM] DomU terrible network performance trought NAT X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 12:06:24 -0000 Le 12/05/2011 19:36, Michael MacLeod a écrit : > Try -rxcsum as well. I have both disabled on my domU. I tried with -rxcsum as you mentionned, but unfortunately it changed nothing on the performance. > > On Thu, May 12, 2011 at 12:53 PM, laurent.cligny@gmail.com > > wrote: > > Le 12/05/2011 17:02, Tobias P. Santos a écrit : > > Try this: > > > > ifconfig xn1 -txcsum > > > > Best regards, > > Tobias. > Hello Tobias, > > Unfortunately ifconfig xn1 -txcsum didn't give me lot of result. > Doing this my network perf through NAT is about 7KiB/s (that is better > than before, but still unworkable). > > Thanks for the idea, maybe we are not that far away from the solution. > > > > > > > Laurent Cligny wrote: > >> Hello all, > >> > >> I want to setup a FreeBSD DomU VM as a network NAT gateway in > order to > >> provide Internet access to other FreeBSD and Linux DomU VMs. > >> My Dom0 is Xen Cloud Platform 1.0 on a Dell Poweredge 210 with > 8 cores, > >> 16GiB RAM and one NIC. > >> > >> All FreeBSD VM are 8.2 amd64 with XENHVM kernel anf the Linux > VM is a > >> Paravirtualized Debian amd64. > >> > >> The NAT gateway is a two-NIC box, with one (xn0) configured with a > >> public Internet address and the other (xn1) configured with a > class A > >> private address (10.0.0.254). > >> The Internet traffic is very good from my FreeBSD NAT gateway > to the > >> Internet (~ 50MiB/s) and also between other VMs on the private > network > >> (~ 70MiB/s trough scp) after applying the patch here > >> (http://www.mail-archive.com/freebsd-xen@freebsd.org/msg00855.html) > >> fixing the "too many frags" problem. > >> > >> Here is a schema of my setup with the observed net speed on it: > >> > >> ---------- --------------- ---------------------- > >> |Internet|---xn0---|FBSD DomU NAT|---xn1---|FBSD and Linux DomUs| > >> ---------- --------------- ---------------------- > >> <-------> <-------> > >> 50MiB/s 70MiB/s > >> > >> <-------------------------------> > >> 0KiB/s >> > >> [...] > > _______________________________________________ > > freebsd-xen@freebsd.org mailing > list > > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > > To unsubscribe, send any mail to > "freebsd-xen-unsubscribe@freebsd.org > " > > _______________________________________________ > freebsd-xen@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > To unsubscribe, send any mail to > "freebsd-xen-unsubscribe@freebsd.org > " > > From owner-freebsd-xen@FreeBSD.ORG Wed May 18 12:09:32 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D44E7106566B for ; Wed, 18 May 2011 12:09:32 +0000 (UTC) (envelope-from laurent.cligny@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 277048FC08 for ; Wed, 18 May 2011 12:09:31 +0000 (UTC) Received: by ewy1 with SMTP id 1so636036ewy.13 for ; Wed, 18 May 2011 05:09:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type; bh=2UHOtnjLoWFsjGOAkKzkK+Xlvz0a08WGsFyg8H0I5Zo=; b=Z9mJrIUWCVmWMsLveI92k6slyxJ+e7xr+PTrhaPJ5ln0in4XlsSoaDL1U7wImMSdfn yoH598wKr+OGzH384fBzdCxkYF3YS8rqtufRWxQm5qIg0oQC+rzS3E+hABXlGtHMCfO/ +9/5F+V9agDHzXjGKi7N3uRronJ69sMfKwO0g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type; b=YjDe9sSqml3FUkF9p7Kp4lrT/VHZRap8mBs9uOWNBW2pZkmfaU1ilGV4GgiN1D6nWR 9Xh6ZH75i31N31ACBZ1CCEZCzjNSA4wjzJdXO5iovuwsBDs2ZEH+iwMLcUtdy1sdYgGt sV2S8VzD2eYBm+XJx65fO0SGc+K6DL6lm7HkA= Received: by 10.213.27.10 with SMTP id g10mr672130ebc.135.1305720570741; Wed, 18 May 2011 05:09:30 -0700 (PDT) Received: from [192.168.1.12] (ARennes-555-1-114-211.w90-49.abo.wanadoo.fr [90.49.97.211]) by mx.google.com with ESMTPS id y17sm1114259eeh.26.2011.05.18.05.09.28 (version=SSLv3 cipher=OTHER); Wed, 18 May 2011 05:09:29 -0700 (PDT) Message-ID: <4DD3B6FD.3060203@gmail.com> Date: Wed, 18 May 2011 14:09:33 +0200 From: "laurent.cligny@gmail.com" User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110402 Iceowl/1.0b2 Icedove/3.1.9 MIME-Version: 1.0 To: Guillaume Seigneuret References: <4DCBEEE0.9060705@steadinet.fr> <4DCBF681.7070106@ntelecom.com.br> <4DCC1097.4080003@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-xen@freebsd.org Subject: Re: [FreeBSD 8.2 amd64 XENHVM] DomU terrible network performance trought NAT X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 12:09:32 -0000 Le 12/05/2011 20:20, Guillaume Seigneuret a écrit : > I think you should try totally disable the window sizing both parts to > isolate it doesn't come from here : > On your FreeBSD VM : sysctl net.inet.tcp.rfc1323=0 > On your Linux box : sysctl net.ipv4.tcp_window_scaling=0 I set net.inet.tcp.rfc1323=0 to VM Nic and the two NAT VM Nics, but it changed nothing on the network perf. > > Then if the rate is hugely better, try some settings to increase the > compatibility. > You can also try an *iperf *test with TCP and UDP packets to see if > the behavior is generalized, and see what latency you have both part > of the network. Iperf is hanging on the test since there is almost not network troughput. I waited 30min before I cancelled it. > > Cordialement, > > *Guillaume Seigneuret* > > > > **Network and System Security Architect* > *Mobile *: +33.6.15.37.35.67 > *Web* : http://www.omegacube.fr > *Address* :* > Hôtel Technologique Marseille Innovation - BP 100 > 45 rue Frederic Joliot Curie > Technopôle de Château Gombert > 13382 Marseille Cedex 13 > > > > 2011/5/12 Michael MacLeod > > > Try -rxcsum as well. I have both disabled on my domU. > > On Thu, May 12, 2011 at 12:53 PM, laurent.cligny@gmail.com > < > laurent.cligny@gmail.com > wrote: > > > Le 12/05/2011 17:02, Tobias P. Santos a écrit : > > > Try this: > > > > > > ifconfig xn1 -txcsum > > > > > > Best regards, > > > Tobias. > > Hello Tobias, > > > > Unfortunately ifconfig xn1 -txcsum didn't give me lot of result. > > Doing this my network perf through NAT is about 7KiB/s (that is > better > > than before, but still unworkable). > > > > Thanks for the idea, maybe we are not that far away from the > solution. > > > > > > > > > > > Laurent Cligny wrote: > > >> Hello all, > > >> > > >> I want to setup a FreeBSD DomU VM as a network NAT gateway in > order to > > >> provide Internet access to other FreeBSD and Linux DomU VMs. > > >> My Dom0 is Xen Cloud Platform 1.0 on a Dell Poweredge 210 > with 8 cores, > > >> 16GiB RAM and one NIC. > > >> > > >> All FreeBSD VM are 8.2 amd64 with XENHVM kernel anf the Linux > VM is a > > >> Paravirtualized Debian amd64. > > >> > > >> The NAT gateway is a two-NIC box, with one (xn0) configured > with a > > >> public Internet address and the other (xn1) configured with a > class A > > >> private address (10.0.0.254). > > >> The Internet traffic is very good from my FreeBSD NAT gateway > to the > > >> Internet (~ 50MiB/s) and also between other VMs on the > private network > > >> (~ 70MiB/s trough scp) after applying the patch here > > >> > (http://www.mail-archive.com/freebsd-xen@freebsd.org/msg00855.html) > > >> fixing the "too many frags" problem. > > >> > > >> Here is a schema of my setup with the observed net speed on it: > > >> > > >> ---------- --------------- ---------------------- > > >> |Internet|---xn0---|FBSD DomU NAT|---xn1---|FBSD and Linux DomUs| > > >> ---------- --------------- ---------------------- > > >> <-------> <-------> > > >> 50MiB/s 70MiB/s > > >> > > >> <-------------------------------> > > >> 0KiB/s > >> > > >> [...] > > > _______________________________________________ > > > freebsd-xen@freebsd.org > mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > > > To unsubscribe, send any mail to > "freebsd-xen-unsubscribe@freebsd.org > " > > > > _______________________________________________ > > freebsd-xen@freebsd.org mailing > list > > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > > To unsubscribe, send any mail to > "freebsd-xen-unsubscribe@freebsd.org > " > > > _______________________________________________ > freebsd-xen@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > To unsubscribe, send any mail to > "freebsd-xen-unsubscribe@freebsd.org > " > > From owner-freebsd-xen@FreeBSD.ORG Wed May 18 12:18:30 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FBA9106566C for ; Wed, 18 May 2011 12:18:30 +0000 (UTC) (envelope-from laurent.cligny@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id CA4458FC13 for ; Wed, 18 May 2011 12:18:29 +0000 (UTC) Received: by ewy1 with SMTP id 1so639221ewy.13 for ; Wed, 18 May 2011 05:18:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=IAspzRIEY68ivC2x+SKiBY2gfnnSUNOO2/QPrJPKfs8=; b=AQ6S6tiN5/NKf86O9c2PHt/uHp66zuNgrFuaOYgqPiLAd3v7fx32ZDoN5Xy5UkeOkB 0yw7NCQTNuTqFGwgCg2enKTwp1f+YLIfpHzOgGKG5PiFLRiZO08xE39DRhfX+EMvm75a k1p6j3r+yHMykBCEbKiQy5WDc+HsTTjiqahpI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; b=j+LpENIAd2lb34FWYEwwRKEfGz29wZ6yX4S+QcTklsRdDmV+QOZq1HG7Lha3pIuQtY cE3IQPD3MZWOnJktFw/MwcOngYRmAyj8oa8rCdU7kL3O/tqNz0ohOHwti92bZ29sMQAb QOrITPyz/2SMFb1iAfxd5RRFqXtXYT9VsAUO0= Received: by 10.14.52.65 with SMTP id d41mr671759eec.85.1305721108538; Wed, 18 May 2011 05:18:28 -0700 (PDT) Received: from [192.168.1.12] (ARennes-555-1-114-211.w90-49.abo.wanadoo.fr [90.49.97.211]) by mx.google.com with ESMTPS id r12sm1124348eeb.4.2011.05.18.05.18.26 (version=SSLv3 cipher=OTHER); Wed, 18 May 2011 05:18:27 -0700 (PDT) Message-ID: <4DD3B917.9080109@gmail.com> Date: Wed, 18 May 2011 14:18:31 +0200 From: "laurent.cligny@gmail.com" User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110402 Iceowl/1.0b2 Icedove/3.1.9 MIME-Version: 1.0 To: freebsd-xen@freebsd.org References: <4DCBEEE0.9060705@steadinet.fr> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Subject: Re: [FreeBSD 8.2 amd64 XENHVM] DomU terrible network performance trought NAT X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 12:18:30 -0000 Le 12/05/2011 17:48, Janne Snabb a écrit : > On Thu, 12 May 2011, Laurent Cligny wrote: > >> All FreeBSD VM are 8.2 amd64 with XENHVM kernel anf the Linux VM is a >> Paravirtualized Debian amd64. > My suggestion would be to try out the same setup with GENERIC kernel > and the rtl driver (or even better e1000 if your Xen allows it) > which is easy to do to make a simple comparison. I tried it following your idea. I compiled GENERIC kernel on both NAT Vm and Vm behind the NAT, the network perf increased a lot as it is now about 15Mb/s trough NAT and between the VM on the private network. But doing this also eat a lot of CPU on "Sys" context (almost 50% on SMP). So I think it is not a suitable setup for me. > In one of my recent benchmarks the FreeBSD Xen PV network driver > performed very well in one direction, but very badly in the other. > Using rtl or e1000 gave much better TCP throughput if you care > equally about both directions. This benchmark was done only for > internal TCP traffic between a Linux dom0 and FreeBSD dumU, the > traffic never entered a real NIC. In fact I have great troughput between VM on private NIC with XENHVM kernel (50 to 70 Mb/s) and also from NAT VM to Internet on public NIC. The problem appears when I generate traffic from VMs on private network to and from the Internet trough the NAT VM. > This might or might not help in your situation... It has conforted me into the idea that XENHVM kernel doesn't work well with FreeBSD NAT solutions or with the underlying Xen Cloud Platform Dom0. > -- > Janne Snabb / EPIPE Communications > snabb@epipe.com - http://epipe.com/ > _______________________________________________ > freebsd-xen@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org" From owner-freebsd-xen@FreeBSD.ORG Wed May 18 12:24:26 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85761106578F for ; Wed, 18 May 2011 12:24:26 +0000 (UTC) (envelope-from laurent.cligny@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 058AB8FC0C for ; Wed, 18 May 2011 12:24:25 +0000 (UTC) Received: by eyg7 with SMTP id 7so641361eyg.13 for ; Wed, 18 May 2011 05:24:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=yem9WuJ5+jr94k8KZgUgGDIREinz1wY6gqGM2Ki9LUs=; b=fVAby+PSaR4uidh9AlGc/+v1rQHFWnF3BbLGWwjHwoHVfNAVsV0BuABkbXMfTgdIEN tlDcx9ca42ScZdaK2j2f4tzs+x0CYIaInvCPVe53htODxV6arxi2mWH3DP58ZOPa4++W j4QnOXiztWTJ/CoYThboUTani122hs5Hiq9ss= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; b=nU9ttbMaURWBx5hY3i1WmogsZO67upcqqL80Dk+3kd71ltUNrseU/XtsHXVG/G8bBB WcJR5JRTqKRHC2dTX2KDuckIV59oOVrTezn2iNTvXu5rqvGtnOBfGHRq7iAL+rWJThuq mo1u2Qr5plME6Z/gsNLrZhiG9pcXGH8oRTm5I= Received: by 10.213.17.136 with SMTP id s8mr1519364eba.92.1305721463820; Wed, 18 May 2011 05:24:23 -0700 (PDT) Received: from [192.168.1.12] (ARennes-555-1-114-211.w90-49.abo.wanadoo.fr [90.49.97.211]) by mx.google.com with ESMTPS id y20sm1125771eeh.16.2011.05.18.05.24.21 (version=SSLv3 cipher=OTHER); Wed, 18 May 2011 05:24:22 -0700 (PDT) Message-ID: <4DD3BA7A.7040504@gmail.com> Date: Wed, 18 May 2011 14:24:26 +0200 From: "laurent.cligny@gmail.com" User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110402 Iceowl/1.0b2 Icedove/3.1.9 MIME-Version: 1.0 To: freebsd-xen@freebsd.org References: <4DCBEEE0.9060705@steadinet.fr> <4DCBF681.7070106@ntelecom.com.br> <4DCC1097.4080003@gmail.com> <4DCC2564.8090002@ntelecom.com.br> In-Reply-To: <4DCC2564.8090002@ntelecom.com.br> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Subject: Re: [FreeBSD 8.2 amd64 XENHVM] DomU terrible network performance trought NAT X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 12:24:26 -0000 Le 12/05/2011 20:22, Tobias P. Santos a écrit : > laurent.cligny@gmail.com wrote: >> Le 12/05/2011 17:02, Tobias P. Santos a écrit : >> >>> Try this: >>> >>> ifconfig xn1 -txcsum >>> >>> Best regards, >>> Tobias. >>> >> Hello Tobias, >> >> Unfortunately ifconfig xn1 -txcsum didn't give me lot of result. >> Doing this my network perf through NAT is about 7KiB/s (that is better >> than before, but still unworkable). >> >> Thanks for the idea, maybe we are not that far away from the solution. >> >> > We had a problem where traffic between VMs was too slow (about 5Mbps > on a Gigabit network) and disabling TX checksum solved the problem > (actually, speed increased to more than 1Gbps). > Your problem is different as you have a nice throughput between VMs on > xn1, sorry that I missed that on my first message. > > Did you try to transfer data from something connected to xn0 to one of > yours DomU servers connected to xn1 without NAT? > You also mentioned pf. Did you make any test using natd or anything else? > Something tells me that's not a nic problem, but that's just a guess. I already tried setup with natd, as I found on the list archives that PF can pose problems with XENHVM kernels, but the network perfomance was the same. > > Anyway, on the coming weeks I will have to set up a similar > environment to yours, so maybe I'll face the same problem (although we > use natd daemon instead of pf). > > Good luck! I think I will keep my Linux PV VM to do NAT, as it is the only thing on my Xen setup that cause problems for now. With that I will still be able to work with FreeBSD for the backend servers. > > _______________________________________________ > freebsd-xen@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org" From owner-freebsd-xen@FreeBSD.ORG Fri May 20 11:35:25 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3B10106566B for ; Fri, 20 May 2011 11:35:24 +0000 (UTC) (envelope-from bouyer@asim.lip6.fr) Received: from osiris.lip6.fr (osiris.lip6.fr [IPv6:2001:660:3302:283c::1e]) by mx1.freebsd.org (Postfix) with ESMTP id 669C38FC1E for ; Fri, 20 May 2011 11:35:24 +0000 (UTC) Received: from antioche.lip6.fr (antioche.ipv6.lip6.fr [IPv6:2001:660:3302:282a:204:75ff:fe9f:9e11]) by osiris.lip6.fr (8.14.4/lip6) with ESMTP id p4KBZJnG025321 ; Fri, 20 May 2011 13:35:20 +0200 (CEST) X-pt: osiris.lip6.fr Received: from armandeche.soc.lip6.fr (armandeche.lip6.fr [132.227.63.133]) by antioche.lip6.fr (8.14.4/8.12.11) with ESMTP id p4KBZJFg015193; Fri, 20 May 2011 13:35:19 +0200 (MEST) Received: by armandeche.soc.lip6.fr (Postfix, from userid 20331) id F164534A9C; Fri, 20 May 2011 13:35:18 +0200 (MEST) Date: Fri, 20 May 2011 13:35:18 +0200 From: Manuel Bouyer To: Hugo Silva Message-ID: <20110520113518.GA9031@asim.lip6.fr> References: <4DB7616F.9060109@barafranca.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4DB7616F.9060109@barafranca.com> User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.2.6 (osiris.lip6.fr [IPv6:2001:660:3302:283c::1e]); Fri, 20 May 2011 13:35:20 +0200 (CEST) X-Scanned-By: MIMEDefang 2.71 on IPv6:2001:660:3302:283c::1e Cc: freebsd-xen@freebsd.org, port-xen@NetBSD.org Subject: Re: FreeBSD HVM+PV boot hangs with phy: run_interrupt_driven_hooks: still waiting after 60 seconds for xenbusb_nop_confighook_cb X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2011 11:35:25 -0000 On Wed, Apr 27, 2011 at 01:21:03AM +0100, Hugo Silva wrote: > Hello, > > Just as the subject says. > > dom0 is NetBSD 5.99.49/Xen 3.3.2 and domU is FreeBSD 8.2-STABLE (same > happens with -RELEASE). I tried to boot a FreeBSD 8.2-RELEASE on both Xen 3.3 and 4.1, the HVMPV kernel dies with a panic in the netfront driver (the panic is "do something smart"). I got 2 sightly different console output; one being clearly that it couldn't read the mac address (I suspect it's trying to read it from the wrong xenstore path), I guess the other case is that it can't get allocate a grant ref. > > If I create a FreeBSD VM and use file:, XENHVM will boot to completion. > > If I vnconfig that same file, then it will hang. On lvm storage it will > also hang. > > So it seems to be related to using phy:. It could be because with file:, the backend driver uses a vnd device attached to the file while qemu-dm opens the file directly. In the phy: case, both qemu-dm and the backend driver wants to open the device, and this may be denied by the kernel. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference -- From owner-freebsd-xen@FreeBSD.ORG Fri May 20 11:39:05 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CBD3106566C for ; Fri, 20 May 2011 11:39:05 +0000 (UTC) (envelope-from bouyer@asim.lip6.fr) Received: from osiris.lip6.fr (osiris.lip6.fr [IPv6:2001:660:3302:283c::1e]) by mx1.freebsd.org (Postfix) with ESMTP id 7F3D08FC16 for ; Fri, 20 May 2011 11:39:04 +0000 (UTC) Received: from antioche.lip6.fr (antioche.ipv6.lip6.fr [IPv6:2001:660:3302:282a:204:75ff:fe9f:9e11]) by osiris.lip6.fr (8.14.4/lip6) with ESMTP id p4KBd2Ts022828 ; Fri, 20 May 2011 13:39:02 +0200 (CEST) X-pt: osiris.lip6.fr Received: from armandeche.soc.lip6.fr (armandeche.lip6.fr [132.227.63.133]) by antioche.lip6.fr (8.14.4/8.12.11) with ESMTP id p4KBd2cA004229; Fri, 20 May 2011 13:39:02 +0200 (MEST) Received: by armandeche.soc.lip6.fr (Postfix, from userid 20331) id 0EBCA34A9C; Fri, 20 May 2011 13:39:02 +0200 (MEST) Date: Fri, 20 May 2011 13:39:02 +0200 From: Manuel Bouyer To: Hugo Silva Message-ID: <20110520113901.GA12239@asim.lip6.fr> References: <4DB7616F.9060109@barafranca.com> <20110520113518.GA9031@asim.lip6.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110520113518.GA9031@asim.lip6.fr> User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.2.6 (osiris.lip6.fr [IPv6:2001:660:3302:283c::1e]); Fri, 20 May 2011 13:39:02 +0200 (CEST) X-Scanned-By: MIMEDefang 2.71 on IPv6:2001:660:3302:283c::1e Cc: freebsd-xen@freebsd.org, port-xen@NetBSD.org Subject: Re: FreeBSD HVM+PV boot hangs with phy: run_interrupt_driven_hooks: still waiting after 60 seconds for xenbusb_nop_confighook_cb X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2011 11:39:05 -0000 On Fri, May 20, 2011 at 01:35:18PM +0200, Manuel Bouyer wrote: > On Wed, Apr 27, 2011 at 01:21:03AM +0100, Hugo Silva wrote: > > Hello, > > > > Just as the subject says. > > > > dom0 is NetBSD 5.99.49/Xen 3.3.2 and domU is FreeBSD 8.2-STABLE (same > > happens with -RELEASE). > > I tried to boot a FreeBSD 8.2-RELEASE on both Xen 3.3 and 4.1, the > HVMPV kernel dies with a panic in the netfront driver (the panic > is "do something smart"). I got 2 sightly different console output; > one being clearly that it couldn't read the mac address (I suspect it's trying > to read it from the wrong xenstore path), I guess the other case is that it > can't get allocate a grant ref. I found it: for HVMPV one has to remove 'ioemu' from the vif config. Now I can look at the block device issue. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference -- From owner-freebsd-xen@FreeBSD.ORG Fri May 20 11:46:35 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 123291065670 for ; Fri, 20 May 2011 11:46:35 +0000 (UTC) (envelope-from snabb@epipe.com) Received: from tiktik.epipe.com (tiktik.epipe.com [IPv6:2001:1828:0:3::2]) by mx1.freebsd.org (Postfix) with ESMTP id C9B638FC22 for ; Fri, 20 May 2011 11:46:34 +0000 (UTC) Received: from tiktik.epipe.com (tiktik.epipe.com [IPv6:2001:1828:0:3::2]) by tiktik.epipe.com (8.14.4/8.14.4) with ESMTP id p4KBkPu0073755 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 20 May 2011 11:46:25 GMT (envelope-from snabb@epipe.com) X-DKIM: Sendmail DKIM Filter v2.8.3 tiktik.epipe.com p4KBkPu0073755 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=epipe.com; s=default; t=1305891985; x=1306496785; bh=PBH4XdO7Q6udRSE0vI76BkKXiHXPv8Z+4V0kCyjUw3I=; h=Date:From:To:cc:Subject:In-Reply-To:Message-ID:References: MIME-Version:Content-Type; b=M9MPk1XWNkIbhZFa8dKAQxVerBBOmuwQKjT977OnO4bWcw8H3mbSigO2Qc+uYTdbP L7Xh2j2Wk0Nw70wBuKOzsNm220OQVoJRRWY4MbK8sl7jccLTjVvxm5ALe5eQHXOfwo Luxa5XXvr1WACW1kWjU6EBU6yRQ8uwjcLc5OECvY= Date: Fri, 20 May 2011 11:46:25 +0000 (UTC) From: Janne Snabb To: Manuel Bouyer In-Reply-To: <20110520113901.GA12239@asim.lip6.fr> Message-ID: References: <4DB7616F.9060109@barafranca.com> <20110520113518.GA9031@asim.lip6.fr> <20110520113901.GA12239@asim.lip6.fr> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (tiktik.epipe.com [IPv6:2001:1828:0:3::2]); Fri, 20 May 2011 11:46:25 +0000 (UTC) Cc: freebsd-xen@freebsd.org, port-xen@NetBSD.org Subject: Re: FreeBSD HVM+PV boot hangs with phy: run_interrupt_driven_hooks: still waiting after 60 seconds for xenbusb_nop_confighook_cb X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2011 11:46:35 -0000 On Fri, 20 May 2011, Manuel Bouyer wrote: > I found it: for HVMPV one has to remove 'ioemu' from the vif config. > Now I can look at the block device issue. Yeah, see: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/154302 Hopefully someone can commit a fix one day... lots of people are hitting their head on this. -- Janne Snabb / EPIPE Communications snabb@epipe.com - http://epipe.com/ From owner-freebsd-xen@FreeBSD.ORG Fri May 20 12:01:48 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AA8B1065698 for ; Fri, 20 May 2011 12:01:48 +0000 (UTC) (envelope-from bouyer@asim.lip6.fr) Received: from osiris.lip6.fr (osiris.lip6.fr [IPv6:2001:660:3302:283c::1e]) by mx1.freebsd.org (Postfix) with ESMTP id B97B18FC19 for ; Fri, 20 May 2011 12:01:47 +0000 (UTC) Received: from antioche.lip6.fr (antioche.ipv6.lip6.fr [IPv6:2001:660:3302:282a:204:75ff:fe9f:9e11]) by osiris.lip6.fr (8.14.4/lip6) with ESMTP id p4KC1juH019946 ; Fri, 20 May 2011 14:01:45 +0200 (CEST) X-pt: osiris.lip6.fr Received: from armandeche.soc.lip6.fr (armandeche.lip6.fr [132.227.63.133]) by antioche.lip6.fr (8.14.4/8.12.11) with ESMTP id p4KC1jRX024438; Fri, 20 May 2011 14:01:45 +0200 (MEST) Received: by armandeche.soc.lip6.fr (Postfix, from userid 20331) id 9945F34A9C; Fri, 20 May 2011 14:01:45 +0200 (MEST) Date: Fri, 20 May 2011 14:01:45 +0200 From: Manuel Bouyer To: Hugo Silva Message-ID: <20110520120145.GA1015@asim.lip6.fr> References: <4DB7616F.9060109@barafranca.com> <20110520113518.GA9031@asim.lip6.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110520113518.GA9031@asim.lip6.fr> User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.2.6 (osiris.lip6.fr [IPv6:2001:660:3302:283c::1e]); Fri, 20 May 2011 14:01:45 +0200 (CEST) X-Scanned-By: MIMEDefang 2.71 on IPv6:2001:660:3302:283c::1e Cc: freebsd-xen@freebsd.org, port-xen@NetBSD.org Subject: Re: FreeBSD HVM+PV boot hangs with phy: run_interrupt_driven_hooks: still waiting after 60 seconds for xenbusb_nop_confighook_cb X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2011 12:01:49 -0000 On Fri, May 20, 2011 at 01:35:18PM +0200, Manuel Bouyer wrote: > > > > If I create a FreeBSD VM and use file:, XENHVM will boot to completion. > > > > If I vnconfig that same file, then it will hang. On lvm storage it will > > also hang. > > > > So it seems to be related to using phy:. > > It could be because with file:, the backend driver uses a vnd device attached > to the file while qemu-dm opens the file directly. In the phy: case, both > qemu-dm and the backend driver wants to open the device, and this may be > denied by the kernel. OK, I understand the problem now: the block device backend driver wants a block device (e.g. /dev/vnd0d). qemu-dm wants the raw device (e.g. /dev/rvnd0d). We should pach either qemu-dm to add the 'r' to the path, or xend to do so. My feeling it that's it's easier to do in qemu-dm at this time (we can do a stat(2) first to see if the given path is a block device, and hack the path if it is). -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference -- From owner-freebsd-xen@FreeBSD.ORG Fri May 20 17:13:12 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 215C11065670 for ; Fri, 20 May 2011 17:13:12 +0000 (UTC) (envelope-from bouyer@asim.lip6.fr) Received: from osiris.lip6.fr (osiris.lip6.fr [IPv6:2001:660:3302:283c::1e]) by mx1.freebsd.org (Postfix) with ESMTP id AFD318FC1C for ; Fri, 20 May 2011 17:13:11 +0000 (UTC) Received: from antioche.lip6.fr (antioche.ipv6.lip6.fr [IPv6:2001:660:3302:282a:204:75ff:fe9f:9e11]) by osiris.lip6.fr (8.14.4/lip6) with ESMTP id p4KHD9uZ026815 ; Fri, 20 May 2011 19:13:09 +0200 (CEST) X-pt: osiris.lip6.fr Received: from armandeche.soc.lip6.fr (armandeche.lip6.fr [132.227.63.133]) by antioche.lip6.fr (8.14.4/8.12.11) with ESMTP id p4KHD9fE021329; Fri, 20 May 2011 19:13:09 +0200 (MEST) Received: by armandeche.soc.lip6.fr (Postfix, from userid 20331) id 4C5BA34A9C; Fri, 20 May 2011 19:13:09 +0200 (MEST) Date: Fri, 20 May 2011 19:13:09 +0200 From: Manuel Bouyer To: Hugo Silva Message-ID: <20110520171309.GA6974@asim.lip6.fr> References: <4DB7616F.9060109@barafranca.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4DB7616F.9060109@barafranca.com> User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.2.6 (osiris.lip6.fr [IPv6:2001:660:3302:283c::1e]); Fri, 20 May 2011 19:13:09 +0200 (CEST) X-Scanned-By: MIMEDefang 2.71 on IPv6:2001:660:3302:283c::1e Cc: freebsd-xen@freebsd.org, port-xen@NetBSD.org Subject: Re: FreeBSD HVM+PV boot hangs with phy: run_interrupt_driven_hooks: still waiting after 60 seconds for xenbusb_nop_confighook_cb X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2011 17:13:12 -0000 This problem should now be fixed in xentools33 and 41 (I've successfully booted a FreeBSD HVMPV kernel with phy:/dev/vnd0d). I also integrated haad@'s patch to properly get the partition size. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --