From owner-freebsd-xen@FreeBSD.ORG Mon Feb 17 09:46:29 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B90481B9 for ; Mon, 17 Feb 2014 09:46:29 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 55CE01991 for ; Mon, 17 Feb 2014 09:46:28 +0000 (UTC) Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s1H9kJct083975 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 17 Feb 2014 09:46:20 GMT Date: Mon, 17 Feb 2014 09:46:20 +0000 From: Karl Pielorz To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , freebsd-xen@freebsd.org Subject: Re: Xen PV Networking issue - disable PV NIC in XENHVM FreeBSD? Message-ID: In-Reply-To: <52FA5BEA.6030307@citrix.com> References: <52F8FE87.1010705@citrix.com> <72C34BC121BD5A0FBDAEFA37@Mail-PC.tdx.co.uk> <52FA5BEA.6030307@citrix.com> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2014 09:46:29 -0000 --On 11 February 2014 18:20 +0100 Roger Pau Monn=C3=A9 = =20 wrote: > You could try to disable the following: > > ifconfig xnX -rxcsum -txcsum -tso4 -lro As a follow-up to this, using: ifconfig xnX -rxcsum -txcsum Does actually fix the problem. I finally got a block of time to go through=20 all this stuff again the other day [it's amazing what difference actually=20 having a decent chunk of uninterrupted time makes!] You do need to be careful when setting this - as at one point, it seemed to = cause some kind of issue that may have flooded the network (if it did -=20 this would be the second time we've seen a XenServer DomU flood things by=20 jabbering the same packet over, and over). -Karl From owner-freebsd-xen@FreeBSD.ORG Mon Feb 17 11:07:00 2014 Return-Path: Delivered-To: freebsd-xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B20BFE7B for ; Mon, 17 Feb 2014 11:07:00 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9517411E5 for ; Mon, 17 Feb 2014 11:07:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1HB70tC033303 for ; Mon, 17 Feb 2014 11:07:00 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1HB70rW033300 for freebsd-xen@FreeBSD.org; Mon, 17 Feb 2014 11:07:00 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 17 Feb 2014 11:07:00 GMT Message-Id: <201402171107.s1HB70rW033300@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-xen@FreeBSD.org Subject: Current problem reports assigned to freebsd-xen@FreeBSD.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2014 11:07:00 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/183139 xen [xen] [patch] ifconfig options on xn0 lost after xen v o kern/180788 xen [xen] [panic] XEN PV kernel 9.2-BETA1 panics on boot o kern/180403 xen [xen] Problems with GENERIC and XENHVM kernels with Xe o kern/180402 xen [xen] XEN kernel does not load in XenClient 4.5.5 o kern/179814 xen [xen] mountroot fails with error=19 under Xen on 9-STA o kern/176471 xen [xen] xn driver crash on detach o kern/176053 xen [xen] [patch] i386: Correct wrong usage of vsnprintf() o kern/175954 xen [xen] XENHVM xn network driver extreme packet loss dur o kern/175822 xen [xen] FreeBSD 9.1 does not work with Xen 4.0 o kern/175757 xen [xen] [patch] xen pvhvm looses keyboard input from VNC o kern/171873 xen [xen] xn network device floods warning in dmesg o kern/171118 xen [xen] FreeBSD XENHVM guest doesn't shutdown cleanly o kern/166174 xen [xen] Problems ROOT MOUNT ERROR o kern/165418 xen [xen] Problems mounting root filesystem from XENHVM o kern/164630 xen [xen] XEN HVM kernel: run_interrupt_driven_hooks: stil o kern/164450 xen [xen] Failed to install FreeeBSD 9.0-RELEASE from CD i o kern/162677 xen [xen] FreeBSD not compatible with "Current Stable Xen" o kern/161318 xen [xen] sysinstall crashes with floating point exception o kern/155468 xen [xen] Xen PV i386 multi-kernel CPU system is not worki o kern/155353 xen [xen] [patch] put "nudging TOD" message under boot_ver o kern/154833 xen [xen]: xen 4.0 - DomU freebsd8.2RC3 i386, XEN kernel. o kern/154473 xen [xen] xen 4.0 - DomU freebsd8.1 i386, XEN kernel. Not o kern/154472 xen [xen] xen 4.0 - DomU freebsd8.1 i386 xen kernel reboot o kern/154428 xen [xen] xn0 network interface and PF - Massive performan o kern/153674 xen [xen] i386/XEN idle thread shows wrong percentages o kern/153672 xen [xen] [panic] i386/XEN panics under heavy fork load o kern/153620 xen [xen] Xen guest system clock drifts in AWS EC2 (FreeBS o kern/153477 xen [xen] XEN pmap code abuses vm page queue lock o kern/153150 xen [xen] xen/ec2: disable checksum offloading on interfac o kern/152228 xen [xen] [panic] Xen/PV panic with machdep.idle_mwait=1 o kern/144629 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143398 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143340 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor f kern/143069 xen [xen] [panic] Xen Kernel Panic - Memory modified after f kern/135667 xen ufs filesystem corruption on XEN DomU system f kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc. f kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i p kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all f i386/124516 xen [xen] FreeBSD-CURRENT Xen Kernel Segfaults when config o kern/118734 xen [xen] FreeBSD 6.3-RC1 and FreeBSD 7.0-BETA 4 fail to b 40 problems total. From owner-freebsd-xen@FreeBSD.ORG Mon Feb 17 14:44:14 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D38F8763 for ; Mon, 17 Feb 2014 14:44:14 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 9AB3F1907 for ; Mon, 17 Feb 2014 14:44:13 +0000 (UTC) Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s1HEiCq2021605 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 17 Feb 2014 14:44:12 GMT Date: Mon, 17 Feb 2014 14:44:12 +0000 From: Karl Pielorz To: freebsd-xen@freebsd.org Subject: FreeBSD 10-R 8 vCPU panics at boot under XenServer (on 8 'core' CPU) Message-ID: <6A3B878077F7D071847052C0@Mail-PC.tdx.co.uk> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2014 14:44:14 -0000 Hi, I've got a FreeBSD 10-R amd64 DomU guest I'm using under XenServer 6.2 (SP1) - this was working fine (i.e. had been restarted many times - while I look at things like HAST). I noticed the other day it was only set to use 4 vCPU's - so I increased this to 8 (the machine has an 4 Core, 8 Thread Xeon 1230v3 in it - which Xen see's as 8 CPU cores). However, it won't boot reliably now: " ... SMP: AP CPU #5 Launched panic: can't schedule timer cpuid = 0 KDB: stack backtrace: #0 0xffffffff808e7dd0 at kdb_backtrace+0x60 #1 0xffffffff808af8b5 at panic+0x155 #2 0xffffffff807a14dd at xentimer_et_start+0xed #3 0xffffffff80d66d6d at loadtimer+0xfd #4 0xffffffff80d657fd at handleevents+0x308 #5 0xffffffff80d65fc8 at timercb+0x308 #6 0xffffffff807a152d at xentimer_intr+0x4d #7 0xffffffff80883e5b at intr_event_handle+0x9b ... " Less than 8 vCPU's seems to boot OK (e.g. 7) and 8 vCPU's has booted a couple of times (out of 30+ reboots). The system is running GENERIC with: options NO_ADAPTIVE_MUTEXES options NO_ADAPTIVE_RWLOCKS options NO_ADAPTIVE_SX In addition XenServer is set to pass through the bare machine's LSI and two Intel NIC's (which it does - and are working, once FreeBSD is booted). Any idea's? -Karl From owner-freebsd-xen@FreeBSD.ORG Mon Feb 17 15:56:38 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02B5BFFF for ; Mon, 17 Feb 2014 15:56:38 +0000 (UTC) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 36A251137 for ; Mon, 17 Feb 2014 15:56:36 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,861,1384300800"; d="scan'208";a="103202748" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 17 Feb 2014 15:56:15 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.2.342.4; Mon, 17 Feb 2014 10:56:15 -0500 Message-ID: <5302311E.2040700@citrix.com> Date: Mon, 17 Feb 2014 16:56:14 +0100 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Karl Pielorz , Subject: Re: FreeBSD 10-R 8 vCPU panics at boot under XenServer (on 8 'core' CPU) References: <6A3B878077F7D071847052C0@Mail-PC.tdx.co.uk> In-Reply-To: <6A3B878077F7D071847052C0@Mail-PC.tdx.co.uk> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-DLP: MIA1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2014 15:56:38 -0000 On 17/02/14 15:44, Karl Pielorz wrote: > > Hi, > > I've got a FreeBSD 10-R amd64 DomU guest I'm using under XenServer 6.2 > (SP1) - this was working fine (i.e. had been restarted many times - > while I look at things like HAST). > > I noticed the other day it was only set to use 4 vCPU's - so I increased > this to 8 (the machine has an 4 Core, 8 Thread Xeon 1230v3 in it - which > Xen see's as 8 CPU cores). > > However, it won't boot reliably now: > > " > ... > SMP: AP CPU #5 Launched > panic: can't schedule timer > cpuid = 0 > KDB: stack backtrace: > #0 0xffffffff808e7dd0 at kdb_backtrace+0x60 > #1 0xffffffff808af8b5 at panic+0x155 > #2 0xffffffff807a14dd at xentimer_et_start+0xed > #3 0xffffffff80d66d6d at loadtimer+0xfd > #4 0xffffffff80d657fd at handleevents+0x308 > #5 0xffffffff80d65fc8 at timercb+0x308 > #6 0xffffffff807a152d at xentimer_intr+0x4d > #7 0xffffffff80883e5b at intr_event_handle+0x9b > ... > " > > Less than 8 vCPU's seems to boot OK (e.g. 7) and 8 vCPU's has booted a > couple of times (out of 30+ reboots). I usually do most of my testing on a Xen W3550 (8-ways), with a 8 vCPU guest, and I've never seen this crash before. I've even booted a 12 vCPU guest on this 8-way system, and it was fine. How many guests are you running on this host, and how many vCPUs has each one assigned? > The system is running GENERIC with: > > options NO_ADAPTIVE_MUTEXES > options NO_ADAPTIVE_RWLOCKS > options NO_ADAPTIVE_SX > > In addition XenServer is set to pass through the bare machine's LSI and > two Intel NIC's (which it does - and are working, once FreeBSD is booted). I don't think those modifications have any effect on the timer, but could you try to recompile without the NO_ADAPTIE_* modifications and without any device pass-through? In order to provide more debug info, could you apply the following patch: http://xenbits.xen.org/people/royger/0001-xen-debug-Xen-PV-timer.patch It will expand the panic message a little bit. Also, after applying the patch you can manually edit sys/dev/xen/timer/timer.c and increase NUM_RETRIES to see if that solves the problem. Roger. From owner-freebsd-xen@FreeBSD.ORG Mon Feb 17 16:01:48 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E0C9454; Mon, 17 Feb 2014 16:01:48 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E316111EA; Mon, 17 Feb 2014 16:01:45 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,861,1384300800"; d="scan'208";a="101466418" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 17 Feb 2014 16:01:38 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.2.342.4; Mon, 17 Feb 2014 11:01:37 -0500 Message-ID: <53023260.1070109@citrix.com> Date: Mon, 17 Feb 2014 17:01:36 +0100 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: John Baldwin , Andrew Cooper Subject: Re: [Xen-devel] [PATCH RFC 09/13] xen: change quality of the MADT ACPI enumerator References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <1980951.95r2q2cca3@ralph.baldwin.cx> <52FD7624.90202@citrix.com> <201402141251.10278.jhb@freebsd.org> In-Reply-To: <201402141251.10278.jhb@freebsd.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-DLP: MIA1 Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2014 16:01:48 -0000 On 14/02/14 18:51, John Baldwin wrote: > On Thursday, February 13, 2014 8:49:24 pm Andrew Cooper wrote: >> On 08/02/2014 21:42, John Baldwin wrote: >>> On Tuesday, December 24, 2013 12:20:58 PM Roger Pau Monne wrote: >>>> Lower the quality of the MADT ACPI enumerator, so on Xen Dom0 we can >>>> force the usage of the Xen mptable enumerator even when ACPI is >>>> detected. >>> Hmm, so I think one question is why does the existing MADT parser >>> not work with the MADT table provided by Xen? This may very well >>> be correct, but if it's only a small change to make the existing >>> MADT parser work with Xen's MADT table, that route might be >>> preferable. >>> >> >> For dom0, the MADT seen is the system MADT, which does not bear any >> reality to dom0's topology. For PV domU, no MADT will be found. For >> HVM domU, the MADT seen ought to represent (virtual) reality. > > Hmm, the other changes suggested that you do want to use the I/O APIC > entries and interrupt overrides from the system MADT for dom0? Just > not the CPU entries. Is that correct? Yes, we need the interrupt entries in order to interact with the underlying hardware, but not the CPU entries/topology. Roger. From owner-freebsd-xen@FreeBSD.ORG Mon Feb 17 16:29:43 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 51201448 for ; Mon, 17 Feb 2014 16:29:43 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 0F47D1449 for ; Mon, 17 Feb 2014 16:29:41 +0000 (UTC) Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s1HGTbgd032749 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 17 Feb 2014 16:29:38 GMT Date: Mon, 17 Feb 2014 16:29:36 +0000 From: Karl Pielorz To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , freebsd-xen@freebsd.org Subject: Re: FreeBSD 10-R 8 vCPU panics at boot under XenServer (on 8 'core' CPU) Message-ID: <94A35E308ABE3947989C36EC@Mail-PC.tdx.co.uk> In-Reply-To: <5302311E.2040700@citrix.com> References: <6A3B878077F7D071847052C0@Mail-PC.tdx.co.uk> <5302311E.2040700@citrix.com> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2014 16:29:43 -0000 --On 17 February 2014 16:56 +0100 Roger Pau Monn=C3=A9 = =20 wrote: > How many guests are you running on this host, and how many vCPUs has > each one assigned? Only 1 active guest (the FreeBSD one) - there are others on there, but=20 they're not running (so I'm hoping they don't count? :) > I don't think those modifications have any effect on the timer, but > could you try to recompile without the NO_ADAPTIE_* modifications and > without any device pass-through? Removed the NO_ADAPTIVE_ stuff from the Kernel - I'll have to do the PCI=20 passthrough removes later. > In order to provide more debug info, could you apply the following patch: > > http://xenbits.xen.org/people/royger/0001-xen-debug-Xen-PV-timer.patch > > It will expand the panic message a little bit. Also, after applying the > patch you can manually edit sys/dev/xen/timer/timer.c and increase > NUM_RETRIES to see if that solves the problem. Ok, with that patch applied, removing the NO_ADAPTIVE_* (but like I said -=20 still with the PCI passthroughs in place) I get: " panic: can't schedule timer on vCPU#0, interval: 112847ns " I'll increase NUM_TRIES, try that - then remove the PCI passthrough devices = and give that ago - that'll have to do those in a bit, and post when done. Thanks, -Karl From owner-freebsd-xen@FreeBSD.ORG Mon Feb 17 20:00:33 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 387CFE18 for ; Mon, 17 Feb 2014 20:00:33 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id EEB311A44 for ; Mon, 17 Feb 2014 20:00:32 +0000 (UTC) Received: from study64.tdx.co.uk (study64.tdx.co.uk [62.13.130.231]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s1HK0TD7055716 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 17 Feb 2014 20:00:30 GMT Date: Mon, 17 Feb 2014 20:00:28 +0000 From: Karl Pielorz To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , freebsd-xen@freebsd.org Subject: Re: FreeBSD 10-R 8 vCPU panics at boot under XenServer (on 8 'core' CPU) Message-ID: <1740E0FEE5963358491F4B37@study64.tdx.co.uk> In-Reply-To: <5302311E.2040700@citrix.com> References: <6A3B878077F7D071847052C0@Mail-PC.tdx.co.uk> <5302311E.2040700@citrix.com> X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2014 20:00:33 -0000 --On 17 February 2014 16:56:14 +0100 Roger Pau Monn=C3=A9=20 wrote: > In order to provide more debug info, could you apply the following patch: > > http://xenbits.xen.org/people/royger/0001-xen-debug-Xen-PV-timer.patch > > It will expand the panic message a little bit. Also, after applying the > patch you can manually edit sys/dev/xen/timer/timer.c and increase > NUM_RETRIES to see if that solves the problem. Ok, tried adjusting the NUM_RETRIES #define in that patch (I left the PCI=20 passthroughs in place at the moment). I had no idea what to set it to - so=20 I went for 600. With it set at 600 that same guest now boots Ok now every=20 time I've tried. But I did notice the whole 'SMP AP CPU #x Launched!' takes forever, and=20 varies a lot (e.g. one boot it took nearly 2 minutes to launch all CPU's=20 and continue). I removed the PCI passthroughs on that guest, and it now flies through the=20 AP launches. Unfortunately though I need the passthroughs :( I've passed through the onboard LSI 2308 SAS controller (mps), and a dual=20 port PCI-E Intel NIC (igb) - all the passthroughs work on FreeBSD once it's = booted - but obviously, not without causing the slow AP CPU launches. I also remembered I set 'hw.pci.enable_msi=3D1' and = 'hw.pci.enable_msix=3D0' in=20 /etc/sysctl.conf - someone else found that was necessary to use the LSI in=20 passthrough mode. Aside from the slow launches, do you think (as they work) it's going to=20 cause issues leaving those passthroughs active? Thanks, -Karl From owner-freebsd-xen@FreeBSD.ORG Tue Feb 18 16:46:55 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9AE4044B for ; Tue, 18 Feb 2014 16:46:55 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 3499019FF for ; Tue, 18 Feb 2014 16:46:54 +0000 (UTC) Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s1IGkfTG088879 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 18 Feb 2014 16:46:42 GMT Date: Tue, 18 Feb 2014 16:46:40 +0000 From: Karl Pielorz To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , freebsd-xen@freebsd.org Subject: Re: Xen PV Networking issue - disable PV NIC in XENHVM FreeBSD? Message-ID: <45B62F4EC5BB836857AF0763@Mail-PC.tdx.co.uk> In-Reply-To: References: <52F8FE87.1010705@citrix.com> <72C34BC121BD5A0FBDAEFA37@Mail-PC.tdx.co.uk> <52FA5BEA.6030307@citrix.com> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Feb 2014 16:46:55 -0000 --On 17 February 2014 09:46 +0000 Karl Pielorz wrote: > ifconfig xnX -rxcsum -txcsum > > Does actually fix the problem. Sadly this doesn't. It seems my testing was flawed (unknownly from a VM on another XenServer - which never exhibited the problem). Real users today noticed the issue (trying to connect out from VM's on the same XenServer as the default gateway FreeBSD VM). As replacing that VM with a CentOS PV host fixes the issue, it must be a bug within the FreeBSD PV NIC code somewhere? [I posted a followup post to xs-devel as well, incase anyone else stumbles into the issue and things the rxcsum thing will fix it]. -Karl