From owner-freebsd-xen@FreeBSD.ORG Tue Feb 17 20:16:41 2009 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A08D106564A for ; Tue, 17 Feb 2009 20:16:41 +0000 (UTC) (envelope-from xen@komadev.de) Received: from zim.freshx.de (zim.freshx.de [83.137.99.113]) by mx1.freebsd.org (Postfix) with ESMTP id AC7E08FC08 for ; Tue, 17 Feb 2009 20:16:40 +0000 (UTC) (envelope-from xen@komadev.de) Received: from localhost (localhost.localdomain [127.0.0.1]) by zim.freshx.de (Postfix) with ESMTP id 3EE3E8C01D5; Tue, 17 Feb 2009 21:16:39 +0100 (CET) X-Virus-Scanned: amavisd-new at zim.freshx.de Received: from zim.freshx.de ([127.0.0.1]) by localhost (zim.freshx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P7zvYX-4+Rut; Tue, 17 Feb 2009 21:16:38 +0100 (CET) Received: from zim.freshx.de (zim.freshx.de [192.168.92.113]) by zim.freshx.de (Postfix) with ESMTP id 21BC08C01C2; Tue, 17 Feb 2009 21:16:38 +0100 (CET) Date: Tue, 17 Feb 2009 21:16:38 +0100 (CET) From: xen@komadev.de Sender: kai@freshx.de To: Kip Macy Message-ID: <14830020.49681234901798034.JavaMail.root@zim.freshx.de> In-Reply-To: <13712113.49661234900587705.JavaMail.root@zim.freshx.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [129.132.228.92] X-Mailer: Zimbra 5.0.11_GA_2695.RHEL5 (ZimbraWebClient - FF3.0 (Mac)/5.0.11_GA_2695.RHEL5) Cc: freebsd-xen Subject: Re: domU page faults under heavy network load X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Feb 2009 20:16:41 -0000 Hi, in the kernel i disabled: WITNESS WITNESS_SKIPSPIN in /etc/make.conf i enabled DEBUG_FLAGS=-ggdb To make sure that user/kernel are in sync i just rebuilt/installed both. I furthermore tracked down the problem, since it seems to occur only between the dom0 and its domU, but not between the dom0 and a "real" remote host. FreeBSD 8.0-CURRENT #0 r188718: Tue Feb 17 18:29:43 CET 2009 # dd if=/dev/zero bs=1024k| cpipe -vt | nc dom0 12345 thru: 5500.000ms at 23.3kB/s ( 23.3kB/s avg) 128.0kB thru: 3200.000ms at 40.0kB/s ( 29.4kB/s avg) 256.0kB ... thru: 0.000ms at infGB/s ( 398.8kB/s avg) 5.4MB thru: 0.000ms at infGB/s ( 408.1kB/s avg) 5.5MB Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xd1d9a500 fault code = supervisor read, page not present instruction pointer = 0x21:0xc02e59b0 stack pointer = 0x29:0xc332aba8 frame pointer = 0x29:0xc332abfc code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 1, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (irq137: xn) [thread pid 12 tid 100025 ] Stopped at xlvbd_add+0x2f70: movl 0x900(%edx,%eax,4),%edx Second System (AMD64, CentOS5.2, Xen 3.3.1): .... thru: 0.000ms at infGB/s ( 598.1kB/s avg) 82.9MB thru: 0.000ms at infGB/s ( 599.0kB/s avg) 83.0MB Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xc433a140 fault code = supervisor read, page not present instruction pointer = 0x21:0xc02db38e stack pointer = 0x29:0xc332ab7c frame pointer = 0x29:0xc332ab84 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 1, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (irq137: xn) [thread pid 12 tid 100025 ] Stopped at gnttab_claim_grant_reference+0x2e: movl 0(%eax,%edx,4),%eax Best Kai PS : Between the dom0 and some random system via Gigabit Ethernet i get only 1/4 of the real machine speed # dd if=/dev/zero bs=1024k | cpipe -vt -b 65535 | nc other 12345 thru: 7003.500ms at 9.1MB/s ( 9.1MB/s avg) 64.0MB thru: 3301.650ms at 19.4MB/s ( 12.4MB/s avg) 128.0MB thru: 1600.800ms at 40.0MB/s ( 16.1MB/s avg) 192.0MB thru: 1800.900ms at 35.5MB/s ( 18.7MB/s avg) 256.0MB thru: 2601.300ms at 24.6MB/s ( 19.6MB/s avg) 320.0MB thru: 1400.700ms at 45.7MB/s ( 21.7MB/s avg) 384.0MB thru: 1200.600ms at 53.3MB/s ( 23.7MB/s avg) 448.0MB thru: 1600.800ms at 40.0MB/s ( 25.0MB/s avg) 512.0MB thru: 1500.750ms at 42.6MB/s ( 26.2MB/s avg) 576.0MB thru: 1900.682ms at 33.7MB/s ( 26.8MB/s avg) 640.0MB thru: 2000.035ms at 32.0MB/s ( 27.2MB/s avg) 704.0MB thru: 2300.000ms at 27.8MB/s ( 27.2MB/s avg) 768.0MB thru: 1700.000ms at 37.6MB/s ( 27.8MB/s avg) 832.0MB ----- "Kip Macy" schrieb: > Is this a recent version of HEAD? I'm not able to reproduce. > > <...> > thru: 1947.346ms at 32.9MB/s ( 27.8MB/s avg) 3.3GB > thru: 1946.356ms at 32.9MB/s ( 27.9MB/s avg) 3.4GB > thru: 1946.910ms at 32.9MB/s ( 27.9MB/s avg) 3.4GB > <...> > > Maybe some variation in the options? > > Thanks, > Kip > > On Tue, Feb 10, 2009 at 11:40 AM, Kai Mosebach wrote: > > 1024meg > > > > > > On 2/10/09 8:30 PM, "Kip Macy" wrote: > > > >> How big is the memory allocation to the VM? > >> > >> Thanks, > >> Kip > >> > >> > >> On Tue, Feb 10, 2009 at 7:59 AM, Kai Mosebach > wrote: > >>> Hi, > >>> > >>> running r188436M on Xen3.3.1 (Centos5, x86_64). The reason is not > to clear > >>> but seems related to heavy load on pipes + memory + network? > >>> It is repeatable with : > >>> > >>> bsd8# dd if=/dev/zero | nc otherhost 12345 > >>> > >>> Fatal trap 12: page fault while in kernel mode > >>> cpuid = 0; apic id = 00 > >>> fault virtual address = 0xd216dd00 > >>> fault code = supervisor read, page not present > >>> instruction pointer = 0x21:0xc02e4d40 > >>> stack pointer = 0x29:0xc332aba8 > >>> frame pointer = 0x29:0xc332abfc > >>> code segment = base 0x0, limit 0xfffff, type 0x1b > >>> = DPL 1, pres 1, def32 1, gran 1 > >>> processor eflags = interrupt enabled, resume, IOPL = 0 > >>> current process = 12 (irq137: xn) > >>> [thread pid 12 tid 100025 ] > >>> Stopped at xlvbd_add+0x2f70: movl > 0x900(%edx,%eax,4),%edx > >>> db> bt > >>> > >>> > >>> > >>> On otherhost i get some stuff (between 19 and 62 MB), after nc > closes down > >>> eventually > >>> > >>> otherhost# nc -l 12345 | cpipe -vt -b 65535 >/dev/null > >>> ... > >>> thru: 108930.791ms at 184.3kB/s ( 184.3kB/s avg) 19.6MB > >>> > >>> best Kai > >>> _______________________________________________ > >>> freebsd-xen@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-xen > >>> To unsubscribe, send any mail to > "freebsd-xen-unsubscribe@freebsd.org" > >>> > > > > > >