From owner-freebsd-virtualization@FreeBSD.ORG Mon Sep 23 11:06:54 2013 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 80721CE7 for ; Mon, 23 Sep 2013 11:06:54 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6D0BE2158 for ; Mon, 23 Sep 2013 11:06:54 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r8NB6s1p069640 for ; Mon, 23 Sep 2013 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r8NB6rPJ069638 for freebsd-virtualization@FreeBSD.org; Mon, 23 Sep 2013 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 23 Sep 2013 11:06:53 GMT Message-Id: <201309231106.r8NB6rPJ069638@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 11:06:54 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 7 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon Sep 23 14:21:00 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5089B431; Mon, 23 Sep 2013 14:21:00 +0000 (UTC) (envelope-from osidorkin@gmail.com) Received: from mail-oa0-x234.google.com (mail-oa0-x234.google.com [IPv6:2607:f8b0:4003:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 12BA52119; Mon, 23 Sep 2013 14:21:00 +0000 (UTC) Received: by mail-oa0-f52.google.com with SMTP id n2so555271oag.39 for ; Mon, 23 Sep 2013 07:20:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:cc:content-type; bh=Z20sJMLyL3IMdPb5s7IsYXegRYSKJuFdWvtQyUJ1VAE=; b=bKIOgw9tYeKzkquevOAEMYPrxi9h/rPtU5rHS33Fn5TJ3YpbjGUt8U8IAmbfiULkLz Xdby6UlXWefv4YKzWD7whUevAAKgOuCiqwGs4ekC48C8zf1lAKhmR0rCctsHAVSi18cT Bc2YNAh0VVrWdwu7UXFdzvmp+xdbr8wAhT/x+HrrUzvxmLqqyn6NS9aBUk2mGkX2Gk3K +vTaftNYI46HkBsU16zrGpxWtHL9gDE9Ul8c+V6a6G/6J4V7ya1WuBomdIzhBmJaA7FG tmNXKbAjoC1FQb7J+X4vzAnDmrhoD5J/061vcs6LegaFt2xpmf10WrRw0uaMoeHMgVVc h/vA== X-Received: by 10.182.98.162 with SMTP id ej2mr484369obb.61.1379946059256; Mon, 23 Sep 2013 07:20:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.182.250.100 with HTTP; Mon, 23 Sep 2013 07:20:39 -0700 (PDT) From: Oleg Sidorkin Date: Mon, 23 Sep 2013 18:20:39 +0400 Message-ID: Subject: [Hyper-V][camlock] storvsc driver panics during boot with patches from camlock project To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 Cc: Alexander Motin X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 14:21:00 -0000 Hello. I'm running the latest current (amd64) under Hyper-V with hyper-v services enabled. If camlock patches are applied (http://people.freebsd.org/~mav/camlock_patches/camlock_20130906.patch), I'm hitting the following kernel panic during boot: FreeBSD 10.0-ALPHA2 #5 r255762M: Sun Sep 22 16:48:21 UTC 2013 olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.17-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a Stepping = 7 .... Timecounter "Hyper-V" frequency 10000000 Hz quality 10000000 ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present; to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 10.000 msec storvsc0 on vmbus0 Netvsc initializing... SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #1 Launched! kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 3; apic id = 03 fault virtual address = 0x20 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff804f444c stack pointer = 0x28:0xfffffe011df38610 frame pointer = 0x28:0xfffffe011df38640 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 0 (hv_control_1 taskq) [ thread pid 0 tid 100046 ] Stopped at turnstile_broadcast+0x8c: movq 0x20(%rbx,%rax,1),%rdx db> bt Tracing pid 0 tid 100046 td 0xfffff80001f20490 turnstile_broadcast() at turnstile_broadcast+0x8c/frame 0xfffffe011df38640 __mtx_unlock_sleep() at __mtx_unlock_sleep+0x60/frame 0xfffffe011df38670 unlock_mtx() at unlock_mtx+0x2a/frame 0xfffffe011df38680 _sleep() at _sleep+0x18e/frame 0xfffffe011df38700 cam_periph_runccb() at cam_periph_runccb+0x9e/frame 0xfffffe011df38800 storvsc_attach() at storvsc_attach+0x6d4/frame 0xfffffe011df388a0 device_attach() at device_attach+0x396/frame 0xfffffe011df388f0 hv_vmbus_child_device_register() at hv_vmbus_child_device_register+0xdb/frame 0xfffffe011df38990 vmbus_channel_process_offer() at vmbus_channel_process_offer+0x133/frame 0xfffffe011df389d0 work_item_callback() at work_item_callback+0x26/frame 0xfffffe011df389f0 taskqueue_run_locked() at taskqueue_run_locked+0xe6/frame 0xfffffe011df38a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xa8/frame 0xfffffe011df38a70 fork_exit() at fork_exit+0x9a/frame 0xfffffe011df38ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011df38ab0 --- trap 0, rip = 0, rsp = 0xfffffe011df38b70, rbp = 0 --- db> This patch is not commited yet (CFT thread with changes description is here: http://lists.freebsd.org/pipermail/freebsd-hackers/2013-September/043333.html), but it is going to be commited till the end of the year. As far as I understand, the invocation chain is storvsc_attach->scan_for_luns->cam_periph_runccb Thanks -- Oleg Sidorkin From owner-freebsd-virtualization@FreeBSD.ORG Mon Sep 23 23:04:50 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 926822C3; Mon, 23 Sep 2013 23:04:50 +0000 (UTC) (envelope-from abgupta@microsoft.com) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-by2lp0240.outbound.protection.outlook.com [207.46.163.240]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 50FA5209F; Mon, 23 Sep 2013 23:04:49 +0000 (UTC) Received: from BL2PR03MB210.namprd03.prod.outlook.com (10.255.230.144) by BL2PR03MB211.namprd03.prod.outlook.com (10.255.230.146) with Microsoft SMTP Server (TLS) id 15.0.775.9; Mon, 23 Sep 2013 23:04:42 +0000 Received: from BL2PR03MB210.namprd03.prod.outlook.com ([169.254.1.203]) by BL2PR03MB210.namprd03.prod.outlook.com ([169.254.1.203]) with mapi id 15.00.0775.005; Mon, 23 Sep 2013 23:04:41 +0000 From: "Abhishek Gupta (LIS)" To: Oleg Sidorkin , "freebsd-virtualization@freebsd.org" Subject: RE: [Hyper-V][camlock] storvsc driver panics during boot with patches from camlock project Thread-Topic: [Hyper-V][camlock] storvsc driver panics during boot with patches from camlock project Thread-Index: AQHOuGguv9aR0907K0W1cMZcA3fEAZnT8ZtA Date: Mon, 23 Sep 2013 23:04:41 +0000 Message-ID: <794fb75db92a4df0991a147919727277@BL2PR03MB210.namprd03.prod.outlook.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [2001:4898:80e0:ed43::3] x-forefront-prvs: 09781D4C35 x-forefront-antispam-report: SFV:NSPM; SFS:(189002)(13464003)(377454003)(52604005)(199002)(54356001)(15202345003)(51856001)(65816001)(83072001)(81686001)(19580405001)(83322001)(19580395003)(76796001)(76786001)(74366001)(81816001)(15975445006)(69226001)(80976001)(77096001)(53806001)(56816003)(4396001)(81542001)(46102001)(80022001)(81342001)(47736001)(77982001)(74662001)(56776001)(74502001)(63696002)(76482001)(54316002)(47446002)(79102001)(31966008)(33646001)(49866001)(50986001)(59766001)(47976001)(74876001)(76576001)(74316001)(74706001)(24736002)(3826001); DIR:OUT; SFP:; SCL:1; SRVR:BL2PR03MB211; H:BL2PR03MB210.namprd03.prod.outlook.com; CLIP:2001:4898:80e0:ed43::3; FPR:; RD:InfoNoRecords; MX:1; A:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: DuplicateDomain-a84fc36a-4ed7-4e57-ab1c-3e967bcbad48.microsoft.com Cc: Alexander Motin X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2013 23:04:50 -0000 Hi Oleg, Please give us some time. I shall look at it. Thanks for reporting. Regards, Abhishek -----Original Message----- From: owner-freebsd-virtualization@freebsd.org [mailto:owner-freebsd-virtua= lization@freebsd.org] On Behalf Of Oleg Sidorkin Sent: Monday, September 23, 2013 7:21 AM To: freebsd-virtualization@freebsd.org Cc: Alexander Motin Subject: [Hyper-V][camlock] storvsc driver panics during boot with patches = from camlock project Hello. I'm running the latest current (amd64) under Hyper-V with hyper-v services = enabled. If camlock patches are applied (http://people.freebsd.org/~mav/camlock_patches/camlock_20130906.patch), I'm hitting the following kernel panic during boot: FreeBSD 10.0-ALPHA2 #5 r255762M: Sun Sep 22 16:48:21 UTC 2013 olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 FreeBSD clang vers= ion 3.3 (tags/RELEASE_33/final 183502) 20130610 CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.17-MHz K8-class CPU) Origin =3D "GenuineIntel" Id =3D 0x206a7 Family =3D 0x6 Model =3D 0x2a= Stepping =3D 7 .... Timecounter "Hyper-V" frequency 10000000 Hz quality 10000000 ZFS NOTICE: Pr= efetch is disabled by default if less than 4GB of RAM is present; to enable, add "vfs.zfs.prefetch_disable=3D0" to /boot/loader.c= onf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1= 0.000 msec storvsc0 on vmbus0 Netvsc initializing... SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #1 Launched! kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid =3D 3; apic id =3D 03 fault virtual address =3D 0x20 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff804f444c stack pointer =3D 0x28:0xfffffe011df38610 frame pointer =3D 0x28:0xfffffe011df38640 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D resume, IOPL =3D 0 current process =3D 0 (hv_control_1 taskq) [ thread pid 0 tid 100046 ] Stopped at turnstile_broadcast+0x8c: movq 0x20(%rbx,%rax,1),%= rdx db> bt Tracing pid 0 tid 100046 td 0xfffff80001f20490 turnstile_broadcast() at turnstile_broadcast+0x8c/frame 0xfffffe011df38640 __mtx_unlock_sleep() at __mtx_unlock_sleep+0x60/frame 0xfffffe011df38670 unlock_mtx() at unlock_mtx+0x2a/frame 0xfffffe011df38680 _sleep() at _sleep+0x18e/frame 0xfffffe011df38700 cam_periph_runccb() at cam_periph_runccb+0x9e/frame 0xfffffe011df38800 storvsc_attach() at storvsc_attach+0x6d4/frame 0xfffffe011df388a0 device_attach() at device_attach+0x396/frame 0xfffffe011df388f0 hv_vmbus_child_device_register() at hv_vmbus_child_device_register+0xdb/frame 0xfffffe011df38990 vmbus_channel_process_offer() at vmbus_channel_process_offer+0x133/frame 0xfffffe011df389d0 work_item_callback() at work_item_callback+0x26/frame 0xfffffe011df389f0 taskqueue_run_locked() at taskqueue_run_locked+0xe6/frame 0xfffffe011df38a4= 0 taskqueue_thread_loop() at taskqueue_thread_loop+0xa8/frame 0xfffffe011df38= a70 fork_exit() at fork_exit+0x9a/frame 0xfffffe011df38ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011df38ab0 --- trap 0, rip =3D 0, rsp =3D 0xfffffe011df38b70, rbp =3D 0 --- db> This patch is not commited yet (CFT thread with changes description is here: http://lists.freebsd.org/pipermail/freebsd-hackers/2013-September/043= 333.html), but it is going to be commited till the end of the year. As far as I understand, the invocation chain is storvsc_attach->scan_for_lu= ns->cam_periph_runccb Thanks -- Oleg Sidorkin _______________________________________________ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/ma= ilman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebs= d.org" From owner-freebsd-virtualization@FreeBSD.ORG Tue Sep 24 13:53:44 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 068FC57A for ; Tue, 24 Sep 2013 13:53:44 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id BC2402F42 for ; Tue, 24 Sep 2013 13:53:43 +0000 (UTC) Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id r8ODrZqG093211 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 24 Sep 2013 14:53:35 +0100 (BST) Date: Tue, 24 Sep 2013 14:53:35 +0100 From: Karl Pielorz To: freebsd-virtualization@freebsd.org Subject: PXE booting into a Hyper-V 2012 R2 instance - slows to a crawl? Message-ID: <9779D76A3AD1B8097FFC552A@Mail-PC.tdx.co.uk> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Sep 2013 13:53:44 -0000 Hi, We just updated our Windows Server 2012 test setup here to Windows Server 2012 R2 (i.e. using Hyper-V 2012 R2 now). We are creating new Hyper-V instances using our PXE boot environment (which boots you into a FreeBSD 9.1 amd64 O/S) - we'd build a Hyper-V aware kernel etc. from that (after SVN'ing the latest Hyper-V project source). Hyper-V would take seconds to boot this - but since changing over to R2 it takes a 'very, very long time'. The initial PXE bit happens quickly (i.e. seeing the server, and getting the BTX output out) but then it gets stuck with the \ spinner - it'll spin for a short time, stop, spin again for a short time, stop again. It *does complete* if you leave it - but it takes minutes (e.g. around 5-10 minutes!). This is with creating a 'Generation 1' Hyper-V instance - and replacing the 'Network Card' (which is added by default) with the 'Legacy Network Card'. Anyone noticed anything similar? -Karl From owner-freebsd-virtualization@FreeBSD.ORG Wed Sep 25 14:57:08 2013 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 85EAF7A8 for ; Wed, 25 Sep 2013 14:57:08 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 59F762B75 for ; Wed, 25 Sep 2013 14:57:05 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-245-177.lns20.per2.internode.on.net [121.45.245.177]) (authenticated bits=0) by vps1.elischer.org (8.14.6/8.14.6) with ESMTP id r8PEv08c032599 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Wed, 25 Sep 2013 07:57:03 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5242F9B8.9090300@freebsd.org> Date: Wed, 25 Sep 2013 22:56:56 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: virtualization@FreeBSD.org Subject: Xen (and others Hypervisors) how do they handle IPIs? Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Sep 2013 14:57:08 -0000 If CPUs are mapped around, how are IPIs handled? I assume they must be emulated? I've noticed that under Xen (on both Amazon EC2 and a Redhat server) whenever you schedule a thread it always sits on the run queue for 20 uSecs before it starts running. It looks to me like it's the IPI taking a long time to be emulated. We have some workloads where there is a lot of flipping back and forth between threads and they are slowed down by an order of magnitude due to this.. turning off NOADAPTIVE(mumble) seems to help a bit as some of the reschedules go away, but it's still a problem. Does anyone know if BHyVe or HyperV also have this problem? I have not yet gone to the source of Xen to see what it does but it would be educational to know what the other Hypervisors do. From owner-freebsd-virtualization@FreeBSD.ORG Wed Sep 25 15:05:47 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AC693A11; Wed, 25 Sep 2013 15:05:47 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CA7192C36; Wed, 25 Sep 2013 15:05:46 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.90,978,1371081600"; d="scan'208";a="57286431" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 25 Sep 2013 15:04:35 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.2.342.4; Wed, 25 Sep 2013 11:04:35 -0400 Message-ID: <5242FB81.4090002@citrix.com> Date: Wed, 25 Sep 2013 17:04:33 +0200 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Julian Elischer Subject: Re: Xen (and others Hypervisors) how do they handle IPIs? References: <5242F9B8.9090300@freebsd.org> In-Reply-To: <5242F9B8.9090300@freebsd.org> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-DLP: MIA1 Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Sep 2013 15:05:47 -0000 On 25/09/13 16:56, Julian Elischer wrote: > If CPUs are mapped around, how are IPIs handled? I assume they must be > emulated? > > I've noticed that under Xen (on both Amazon EC2 and a Redhat server) > whenever you schedule a thread it always sits on the run queue for 20 > uSecs before it starts running. It looks to me like it's the IPI taking > a long time to be emulated. This has been improved on the FreeBSD Xen PVHVM port by using PV IPIs instead of the emulated ones, see r255331. It should be faster than the previous emulated implementation. From owner-freebsd-virtualization@FreeBSD.ORG Thu Sep 26 01:48:53 2013 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0AFA67AF for ; Thu, 26 Sep 2013 01:48:53 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D48952413 for ; Thu, 26 Sep 2013 01:48:52 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-245-177.lns20.per2.internode.on.net [121.45.245.177]) (authenticated bits=0) by vps1.elischer.org (8.14.6/8.14.6) with ESMTP id r8Q1mi9x034021 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 25 Sep 2013 18:48:46 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5243927C.9080609@freebsd.org> Date: Thu, 26 Sep 2013 09:48:44 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= Subject: Re: Xen (and others Hypervisors) how do they handle IPIs? References: <5242F9B8.9090300@freebsd.org> <5242FB81.4090002@citrix.com> In-Reply-To: <5242FB81.4090002@citrix.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Sep 2013 01:48:53 -0000 On 9/25/13 11:04 PM, Roger Pau Monné wrote: > On 25/09/13 16:56, Julian Elischer wrote: >> If CPUs are mapped around, how are IPIs handled? I assume they must be >> emulated? >> >> I've noticed that under Xen (on both Amazon EC2 and a Redhat server) >> whenever you schedule a thread it always sits on the run queue for 20 >> uSecs before it starts running. It looks to me like it's the IPI taking >> a long time to be emulated. > This has been improved on the FreeBSD Xen PVHVM port by using PV IPIs > instead of the emulated ones, see r255331. It should be faster than the > previous emulated implementation. > I missed that.. thanks! Do you (or anyone else) know if this can be used on Amazon EC2? And do you need a specific version/configuration of Xen to be able to use it? From owner-freebsd-virtualization@FreeBSD.ORG Thu Sep 26 08:23:26 2013 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9D20D55F; Thu, 26 Sep 2013 08:23:26 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 5F8F0272B; Thu, 26 Sep 2013 08:23:26 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id BD1E11238E; Thu, 26 Sep 2013 18:16:11 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([78.133.116.242]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BOW25026 (AUTH peterg@ptree32.com.au); Thu, 26 Sep 2013 18:16:10 +1000 Message-ID: <5243ED47.8080001@freebsd.org> Date: Thu, 26 Sep 2013 10:16:07 +0200 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Julian Elischer Subject: Re: Xen (and others Hypervisors) how do they handle IPIs? References: <5242F9B8.9090300@freebsd.org> In-Reply-To: <5242F9B8.9090300@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Sep 2013 08:23:26 -0000 Hi Julian, > If CPUs are mapped around, how are IPIs handled? I assume they must be > emulated? For bhyve, if the target vCPU of an IPI is running, a null IPI is sent on the host to force it to exit so the IPI can be injected (vmm.c:vm_interrupt_hostcpu()). If the target is asleep due to being idle, it is woken up. And, if the target is running but not in vCPU context, an interrupt is queued up so it will be injected on the next vmenter (subject to the emulated local APIC interrupt priority) > Does anyone know if BHyVe or HyperV also have this problem? Hmmm, hard to say. Would you be able to try your workload out on a bhyve system ? later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Thu Sep 26 09:50:42 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2760E78A; Thu, 26 Sep 2013 09:50:42 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 46C562EA3; Thu, 26 Sep 2013 09:50:41 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.90,984,1371081600"; d="scan'208";a="57588493" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 26 Sep 2013 09:50:38 +0000 Received: from [127.0.0.1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.2.342.4; Thu, 26 Sep 2013 05:50:38 -0400 Message-ID: <5244036D.7060004@citrix.com> Date: Thu, 26 Sep 2013 11:50:37 +0200 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Julian Elischer Subject: Re: Xen (and others Hypervisors) how do they handle IPIs? References: <5242F9B8.9090300@freebsd.org> <5242FB81.4090002@citrix.com> <5243927C.9080609@freebsd.org> In-Reply-To: <5243927C.9080609@freebsd.org> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-DLP: MIA2 Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Sep 2013 09:50:42 -0000 On 26/09/13 03:48, Julian Elischer wrote: > On 9/25/13 11:04 PM, Roger Pau Monné wrote: >> On 25/09/13 16:56, Julian Elischer wrote: >>> If CPUs are mapped around, how are IPIs handled? I assume they must be >>> emulated? >>> >>> I've noticed that under Xen (on both Amazon EC2 and a Redhat server) >>> whenever you schedule a thread it always sits on the run queue for 20 >>> uSecs before it starts running. It looks to me like it's the IPI taking >>> a long time to be emulated. >> This has been improved on the FreeBSD Xen PVHVM port by using PV IPIs >> instead of the emulated ones, see r255331. It should be faster than the >> previous emulated implementation. >> > I missed that.. thanks! > Do you (or anyone else) know if this can be used on Amazon EC2? > And do you need a specific version/configuration of Xen to be able to > use it? The PV IPIs require Xen version 4.0 or greater, and the PV timer requires 4.0.1 or greater if I'm not mistaken. If you are lucky to get an Amazon instance that uses this Xen version (or any superior one) they will be activated by default, if not FreeBSD will switch to the old event delivery method and PV IPIs and PV timer will be disabled in favour of the emulated ones. I guess it's just a matter of time before Amazon switches all their servers to Xen 4.x. Roger. From owner-freebsd-virtualization@FreeBSD.ORG Thu Sep 26 14:51:32 2013 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D68238CA; Thu, 26 Sep 2013 14:51:32 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AA09E2F19; Thu, 26 Sep 2013 14:51:32 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-245-177.lns20.per2.internode.on.net [121.45.245.177]) (authenticated bits=0) by vps1.elischer.org (8.14.6/8.14.6) with ESMTP id r8QEpOfx036458 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 26 Sep 2013 07:51:30 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <524449E6.4050102@freebsd.org> Date: Thu, 26 Sep 2013 22:51:18 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Xen (and others Hypervisors) how do they handle IPIs? References: <5242F9B8.9090300@freebsd.org> <5243ED47.8080001@freebsd.org> In-Reply-To: <5243ED47.8080001@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Sep 2013 14:51:32 -0000 On 9/26/13 4:16 PM, Peter Grehan wrote: > Hi Julian, > >> If CPUs are mapped around, how are IPIs handled? I assume they must be >> emulated? > > For bhyve, if the target vCPU of an IPI is running, a null IPI is > sent on the host to force it to exit so the IPI can be injected > (vmm.c:vm_interrupt_hostcpu()). > > If the target is asleep due to being idle, it is woken up. And, if > the target is running but not in vCPU context, an interrupt is > queued up so it will be injected on the next vmenter (subject to the > emulated local APIC interrupt priority) > >> Does anyone know if BHyVe or HyperV also have this problem? > > Hmmm, hard to say. Would you be able to try your workload out on a > bhyve system ? possibly in a few weeks.. I'll try remember when the time is right. > > later, > > Peter. > > > From owner-freebsd-virtualization@FreeBSD.ORG Thu Sep 26 22:44:37 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6FD8E6A7 for ; Thu, 26 Sep 2013 22:44:37 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D5CE42FC3 for ; Thu, 26 Sep 2013 22:44:36 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id eo20so1541232lab.34 for ; Thu, 26 Sep 2013 15:44:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=kBJSHStNa2d+poyXyKvddJBNhwR6vlCKZzg5whDebJY=; b=s7kbog9nueKZSMR+rSu0SAqiW0NuD4Rs6+H4JOgxA/hMd/0VYlpl911TBcxAqqT/OV ncgUrKjXOfCLoWRB5mmasUA/XVeflR7rQOPm/e4naaFoLo1lZgS1WW/IU1GzjLGA14wV gzgt8dpVdQWxbUjPLVwE5hzxK6EdULN/0qBjfgdrB2YquR8Tkuy1ZWhN1PCWHMG2QeYN FtVePfYFQFGt7FwVwkRLqB2dRyJTEX/oRW3WDNnpwYmf+yMpwVpVjD3fd7cNPO/pctwq 5ghAddE+UztMCD/p0raH7Veg2XU4agV9ZxJ2lvD+/maJoq3kqV/p3Z62DXJzcCZu/ERz vIHQ== MIME-Version: 1.0 X-Received: by 10.112.146.33 with SMTP id sz1mr5783653lbb.14.1380235474718; Thu, 26 Sep 2013 15:44:34 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.168.136 with HTTP; Thu, 26 Sep 2013 15:44:34 -0700 (PDT) In-Reply-To: <9779D76A3AD1B8097FFC552A@Mail-PC.tdx.co.uk> References: <9779D76A3AD1B8097FFC552A@Mail-PC.tdx.co.uk> Date: Thu, 26 Sep 2013 15:44:34 -0700 X-Google-Sender-Auth: hcXfYZwBCck3l8t0wl0uIjofSeQ Message-ID: Subject: Re: PXE booting into a Hyper-V 2012 R2 instance - slows to a crawl? From: Craig Rodrigues To: Karl Pielorz Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Sep 2013 22:44:37 -0000 On Tue, Sep 24, 2013 at 6:53 AM, Karl Pielorz wrote: > > > We are creating new Hyper-V instances using our PXE boot environment > (which boots you into a FreeBSD 9.1 amd64 O/S) - we'd build a Hyper-V aware > kernel etc. from that (after SVN'ing the latest Hyper-V project source). > PXE involves DHCP and TFTP. I would recommend that you do the following: (1) On the machine hosting your TFTP server, look at the TFTP logs and look for any clues. If your TFTP server is a FreeBSD box, look at /var/log/xferlog. I don't know what logs are available for a Windows TFTP server. (2) In between your booting machine, and your TFTP server, use Wireshark or tcpdump to capture the network traffic and analyze what is going on. (3) By default, the FreeBSD PXE loader uses a TFTP blocksize of 512 bytes per block (which is the original TFTP standard). You can increase the size of that to something bigger, depending on the MTU of your network. Usually 1400 bytes will work. In /boot/defaults/loader.conf, you can see this: #tftp.blksize="1428" # Set the RFC 2348 TFTP block size. # If the TFTP server does not support RFC 2348, # the block size is set to 512. If the value # is out of range ( < 8 || > 9008 ) an error is # returned. If you put in yout /boot/loader.conf file: tftp.blksize="1428" you can see if that speeds up the TFTP transfer during PXE booting. > > Hyper-V would take seconds to boot this - but since changing over to R2 it > takes a 'very, very long time'. > > The initial PXE bit happens quickly (i.e. seeing the server, and getting > the BTX output out) but then it gets stuck with the \ spinner - it'll spin > for a short time, stop, spin again for a short time, stop again. > The spinner is the part where it is loading the kernel. For PXE, this is happening over TFTP. This looks like a slow network problem, so you need to analyze it a bit. I wrote some notes on PXE booting in FreeBSD which you might want to get some ideas from: http://www.freebsd.org/doc/handbook/network-pxe-nfs.html -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 01:59:26 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C6822AD1 for ; Fri, 27 Sep 2013 01:59:26 +0000 (UTC) (envelope-from miguelmclara@gmail.com) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6261C28CC for ; Fri, 27 Sep 2013 01:59:26 +0000 (UTC) Received: by mail-wi0-f169.google.com with SMTP id hj3so216703wib.4 for ; Thu, 26 Sep 2013 18:59:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=yd77qiVELeLBzJP+Wk79DO6+9i321aYR2hO61cxxNb0=; b=S7mqb1+SKi4mbxAMNG6Am/bF5FnKBqzTEXqSuDqPAe5slG2HNHP/rogaw6rBGj7c+H x89ORiQ8/vMIRYSmCcFyte+XjvH74Ut68vrDH+6Osx4wDr86KWVGvVpTmQ0ft7PBZVOB vUMwXIuMRYHrfYidqS4C66aKlsqvhK9G9YHlcdxkKgulo6LFM59jOhVCnNefwnPn86Ic J6GIr7MoLbpfCBEOGrsfHxfSMMRx6V575AgqnRbTh09m5syYzEZJdIFtw/ZZT8HYDBox us5NY+Ou0+3yZLoR0A/V+qm1njM1vBZ4Gy6UmdWDFg+5OZIPIsDiQBNmA+E9adcOkFLe bQPw== X-Received: by 10.180.185.77 with SMTP id fa13mr456012wic.58.1380247164328; Thu, 26 Sep 2013 18:59:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.147.68 with HTTP; Thu, 26 Sep 2013 18:59:04 -0700 (PDT) From: Miguel Clara Date: Fri, 27 Sep 2013 01:59:04 +0000 Message-ID: Subject: bhyve vmrum vm_setup_memory: Inappropriate ioctl for device To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 01:59:26 -0000 Just wanted to start testing with bhyve, but I'm getting this error creating an exmaple VM: # ./vmrun.sh -c 1 vm1 virtio disk device file "./diskdev" does not exist. Creating it ... Launching virtual machine "vm1" ... vm_setup_memory: Inappropriate ioctl for device From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 02:04:09 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 77F0CCB6; Fri, 27 Sep 2013 02:04:09 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [208.86.227.67]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4E61D291B; Fri, 27 Sep 2013 02:04:09 +0000 (UTC) Received: from glenbarber.us (c-71-224-221-174.hsd1.nj.comcast.net [71.224.221.174]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id CE5977EF7; Fri, 27 Sep 2013 02:04:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 mail0.glenbarber.us CE5977EF7 Authentication-Results: mail0.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Thu, 26 Sep 2013 22:04:06 -0400 From: Glen Barber To: Miguel Clara Subject: Re: bhyve vmrum vm_setup_memory: Inappropriate ioctl for device Message-ID: <20130927020406.GY2335@glenbarber.us> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="3A73bywyZFNg8Vvt" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 10.0-ALPHA2 amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 02:04:09 -0000 --3A73bywyZFNg8Vvt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 27, 2013 at 01:59:04AM +0000, Miguel Clara wrote: > Just wanted to start testing with bhyve, but I'm getting this error > creating an exmaple VM: >=20 > # ./vmrun.sh -c 1 vm1 > virtio disk device file "./diskdev" does not exist. > Creating it ... > Launching virtual machine "vm1" ... > vm_setup_memory: Inappropriate ioctl for device What does dmesg say? What is 'uname -a' ? Glen --3A73bywyZFNg8Vvt Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iQIcBAEBCAAGBQJSROeWAAoJELls3eqvi17QFzIP/igZhLkBTj1r+bJWXPONAXEY BTMz7bo+6YjiZrC6ILuRiHgY29ygPNHhdcpCvr0tuSB/eJ+DXcvaNg3Gtqo37+x4 1QseSnaDKCtXSOhC1RUhchR0n23NJ46ba44+Wl3TBFhbC5x/fyVJ7yfrhrox0oBK lEBDaQTssLNBRLHNKjJ2dBhE06UqT0tH/QJ7U/Pbcrf8y/bPXyOCMB8ihObdNsj6 YmwhWBiBRHR+CyD8Us3/cQhTMBjoq+mA44NhU13XGKjvQ1D3MBwnB4pfoIMfNgM+ stjN2+8lmpxmfvV42JEbhlmnken2LNcGlAXCU8QmHzj4cU0y9XJj7QTQ8QQZJkPm zRAzsIKAFR6dNy+tsYkgtvBgvkovwwPLfmFwzimkZ5Utdg1OJHM0X3Q1SOKtr4Ht Ecb5rJB+8q9WjdHjqJo4GTTXNHCIkviRL4avslDhE4wy+Mak92ucOUPst+WW5mjm CCz9lOAhBnvWwOJfQ6g05dgEfpczD6zUm6BoyHdACAgbJ81sXOinoHRhD3sGDcCT IqiXl/LKxjGwHfrSa3J+L6RQ5fsV/liOpoO4YPWqIUxhKT/PBhqoOijJpAzcEg60 TM4LFOY/1wiG9taLHEW7xxv434cqFMRM6T+FMJP0zOBQR/QPb7TR4UpgZvgm7Nqr IR01Xp+6hJY9G7N5p4xP =l3tp -----END PGP SIGNATURE----- --3A73bywyZFNg8Vvt-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 02:10:30 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DDD8AE51; Fri, 27 Sep 2013 02:10:29 +0000 (UTC) (envelope-from miguelmclara@gmail.com) Received: from mail-we0-x22a.google.com (mail-we0-x22a.google.com [IPv6:2a00:1450:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 56550294C; Fri, 27 Sep 2013 02:10:29 +0000 (UTC) Received: by mail-we0-f170.google.com with SMTP id w62so2046538wes.29 for ; Thu, 26 Sep 2013 19:10:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=oCoRLh3iYxchOrRn4/RyAlvycCFtPHTVbOtxvbJs3OI=; b=WeJVTSS3UTR3CeujM+Cm8+0shXBvswKbczjNKLcKXDHwcZx8jyOGYtid0BAtwbi4vG aFoguL0R3RLYNbKk7Na4FoLrXBdR6MRvCiofRXXC2Opi/hfSp5aIVdZJR5gXJ8/TJfVj cwI8zjeCs/ZlBvbs/wZcdXxb5WNFoO47t3P6HzEVDOommVSMeQciQUHFp/a0rj9/1q3D YsO50/618OdBW4E8UIg5mR6aRX713JBJNnFmZqfaB4aLJo+SSFgn6HoB5T9hN+6HqJnO ElLIgOYTxhGEo/cNXtaXvxbYX7bdJq4PDuxG5P6QXVnQ5a+QhCwvpIViD6xg5EBMb+9j qdUw== X-Received: by 10.180.82.164 with SMTP id j4mr407078wiy.65.1380247827690; Thu, 26 Sep 2013 19:10:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.147.68 with HTTP; Thu, 26 Sep 2013 19:10:07 -0700 (PDT) In-Reply-To: <20130927020406.GY2335@glenbarber.us> References: <20130927020406.GY2335@glenbarber.us> From: Miguel Clara Date: Fri, 27 Sep 2013 02:10:07 +0000 Message-ID: Subject: Re: bhyve vmrum vm_setup_memory: Inappropriate ioctl for device To: Glen Barber Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 02:10:30 -0000 Sorry should have posted that info i the first mail! uname -a FreeBSD r2d2 10.0-ALPHA2 FreeBSD 10.0-ALPHA2 #3 r255788M: Mon Sep 23 00:32:50 UTC 2013 root@:/usr/obj/usr/src/sys/GENERIC amd64 dmesg only added an extra line it seams, when I created the tap0, other than that I see nothing! tap0: Ethernet address: 00:bd:05:89:47:00 Thanks On Fri, Sep 27, 2013 at 2:04 AM, Glen Barber wrote: > On Fri, Sep 27, 2013 at 01:59:04AM +0000, Miguel Clara wrote: > > Just wanted to start testing with bhyve, but I'm getting this error > > creating an exmaple VM: > > > > # ./vmrun.sh -c 1 vm1 > > virtio disk device file "./diskdev" does not exist. > > Creating it ... > > Launching virtual machine "vm1" ... > > vm_setup_memory: Inappropriate ioctl for device > > What does dmesg say? > > What is 'uname -a' ? > > Glen > > From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 02:17:24 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6F23AECC; Fri, 27 Sep 2013 02:17:24 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [208.86.227.67]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 42BBC2992; Fri, 27 Sep 2013 02:17:23 +0000 (UTC) Received: from glenbarber.us (c-71-224-221-174.hsd1.nj.comcast.net [71.224.221.174]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id D020D7028; Fri, 27 Sep 2013 02:17:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 mail0.glenbarber.us D020D7028 Authentication-Results: mail0.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Thu, 26 Sep 2013 22:17:21 -0400 From: Glen Barber To: Miguel Clara Subject: Re: bhyve vmrum vm_setup_memory: Inappropriate ioctl for device Message-ID: <20130927021721.GZ2335@glenbarber.us> References: <20130927020406.GY2335@glenbarber.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="nYdblYkROcTNSlQ2" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 10.0-ALPHA2 amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 02:17:24 -0000 --nYdblYkROcTNSlQ2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 27, 2013 at 02:10:07AM +0000, Miguel Clara wrote: > On Fri, Sep 27, 2013 at 2:04 AM, Glen Barber wrote: >=20 > > On Fri, Sep 27, 2013 at 01:59:04AM +0000, Miguel Clara wrote: > > > Just wanted to start testing with bhyve, but I'm getting this error > > > creating an exmaple VM: > > > > > > # ./vmrun.sh -c 1 vm1 > > > virtio disk device file "./diskdev" does not exist. > > > Creating it ... > > > Launching virtual machine "vm1" ... > > > vm_setup_memory: Inappropriate ioctl for device > > > > What does dmesg say? > > > > What is 'uname -a' ? > > > > Sorry should have posted that info i the first mail! >=20 > uname -a > FreeBSD r2d2 10.0-ALPHA2 FreeBSD 10.0-ALPHA2 #3 r255788M: Mon Sep 23 > 00:32:50 UTC 2013 root@:/usr/obj/usr/src/sys/GENERIC amd64 >=20 What are the changes to your kernel config from the GENERIC? Glen > dmesg only added an extra line it seams, when I created the tap0, other > than that I see nothing! >=20 > tap0: Ethernet address: 00:bd:05:89:47:00 >=20 --nYdblYkROcTNSlQ2 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iQIcBAEBCAAGBQJSROqxAAoJELls3eqvi17QhmEP/jN3ymjbPx79lXSr+TpCycyH UV8zNeuTzdT3KpVM4WgIVYYoedia8YpvF6f2r2en8yNG7ibBCRSmPKGmevu6Y8o2 IrmhFMj3aPOEvQsyIU6ymFZd6JMiiUjkLJWx0mq+8FNvBH5spkrU3OZHEmgnUkt9 jJAi5X6RExc3LQy3guTDi0Iv5ipGblA+IxdUgucOch/lKIfJKhbpK+f1rcLlR7He /Gp2s160bkEdK2s13VW/1erAPBocBBq0erS3vHG1hxub/2uorgx6nVilrE54LPcW NTrdroiO8odfiGH5szKwGLcVIfxkYB1ZSvfJuFXkEEYY/5BA8EaejrRTsBYUEVJr w8tiWhpunPaTZ87PDPdhpyQc6rneEL/CFGXWzkEC4xZFq4lZayccqTo7TxQstEuD KkqZUDMJWEDZ8upMsdr5oO+otLdzBjKUx/hZceOj5z8V3DkRJtcTj94y0bVcbI1V mEXJHpjivPn0fmpdio9SsadvTTkvEoF+4cimKqi11ETNRec6SPyLZhnL4NHw8Ou0 MW108VO466acm3JKJ+1/33iiAjW0DMLDslL7QI4E9oZkmALS9LylWQ62JM3p4XMB Vwje3UDSCBffV1+ct5Ik1MjEJNIVCiOt3Ldprlg3YUJ122+3lUm/FdVQJkDTdkPI V4/NVjODbetgFiqUp7wG =ny/2 -----END PGP SIGNATURE----- --nYdblYkROcTNSlQ2-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 02:31:23 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2D7D1163 for ; Fri, 27 Sep 2013 02:31:23 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-qe0-x22e.google.com (mail-qe0-x22e.google.com [IPv6:2607:f8b0:400d:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E303C2A5A for ; Fri, 27 Sep 2013 02:31:22 +0000 (UTC) Received: by mail-qe0-f46.google.com with SMTP id x7so1432771qeu.5 for ; Thu, 26 Sep 2013 19:31:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LPTMV+H9cSUcaBy5brsM1B0/AukMFSK3oJvHfM3htBQ=; b=FYBpQ1zUzlFwpTTgTQBbZc4Kuz68WoWICoqrerqeTNYPyOjYnU1upff+QsJypV7Izh dJeWdOrFbGdUc7P5I5tUFt60iVGt9MQbplQ5nLSwXibqlNb60+y8tnmiM48p8Aoz/ZWm jQOUWjDbLte/kiRHGg8rLE94KUEh03J/zNQe2OqGtkgXanXQYUnL0PQQ/mtyUXis44Bh VfxZ0h/g0ZOcGLRjGVgWVqPHBvfsnylvOl+W+jc9A8AYxtbEuV72Kc1gTMh7abWVE0f8 3d/6MWMBmP17LEkD4SNk5A9d+EqCJsSgYOuqTCJM4iWVjuRjV9e8W7ypxgCHr4hShfpr /YZw== MIME-Version: 1.0 X-Received: by 10.49.130.233 with SMTP id oh9mr6053412qeb.10.1380249082065; Thu, 26 Sep 2013 19:31:22 -0700 (PDT) Received: by 10.140.88.80 with HTTP; Thu, 26 Sep 2013 19:31:21 -0700 (PDT) In-Reply-To: References: Date: Thu, 26 Sep 2013 19:31:21 -0700 Message-ID: Subject: Re: bhyve vmrum vm_setup_memory: Inappropriate ioctl for device From: Neel Natu To: Miguel Clara Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 02:31:23 -0000 Hi Miguel, On Thu, Sep 26, 2013 at 6:59 PM, Miguel Clara wrote: > Just wanted to start testing with bhyve, but I'm getting this error > creating an exmaple VM: > > > > # ./vmrun.sh -c 1 vm1 > virtio disk device file "./diskdev" does not exist. > Creating it ... > Launching virtual machine "vm1" ... > vm_setup_memory: Inappropriate ioctl for device > Your userspace is most likely out of sync with the kernel. Could you try to re-run after a buildworld and installworld? best Neel > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 02:35:48 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E17F6326 for ; Fri, 27 Sep 2013 02:35:48 +0000 (UTC) (envelope-from miguelmclara@gmail.com) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7D9CD2A8A for ; Fri, 27 Sep 2013 02:35:48 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id t60so2062303wes.0 for ; Thu, 26 Sep 2013 19:35:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=eo0Kl6vTvg/0G49J8/nZRrqPOyvrmodgDdszgOLc90g=; b=Qzrq4YGheNspNfJ+I4xybmJ3TjiP8QMBASY2Q0VlT9V8jQjFrmT06O8b2dAIf8qpt7 HfPAiAcaIj5vExWeM4JUkcvzBJzdIxMciRIDBOJEZ3iWOfEKzTCGMVfjWPYo/rypdx20 2wN6d4pw72vC0roIA1Mrpes6EAV0Uo0EVv0N34QB10ReamaekTx1qyGc94ZtvGyTtWUj A8HCaz3qlVHXVAgoG0QQs0G3vXEusKF2wqU7fdDk9zxZzhgsX8di6lXviyJ0L2HNnTls 5bbXA11UqTz4UsD/3a1RGnG1wqId2Dea8lXNTa96ZP6A+c6hl/qXEbte0F3J8AKfyWsj tUHA== X-Received: by 10.180.187.2 with SMTP id fo2mr472655wic.65.1380249346841; Thu, 26 Sep 2013 19:35:46 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.147.68 with HTTP; Thu, 26 Sep 2013 19:35:26 -0700 (PDT) In-Reply-To: References: From: Miguel Clara Date: Fri, 27 Sep 2013 02:35:26 +0000 Message-ID: Subject: Re: bhyve vmrum vm_setup_memory: Inappropriate ioctl for device To: Neel Natu Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 02:35:48 -0000 I was actually just thinking about that... make buildworld is underway... I'll post any updates after the userspace install. Thanks On Fri, Sep 27, 2013 at 2:31 AM, Neel Natu wrote: > Hi Miguel, > > On Thu, Sep 26, 2013 at 6:59 PM, Miguel Clara wrote: > >> Just wanted to start testing with bhyve, but I'm getting this error >> creating an exmaple VM: >> >> >> >> # ./vmrun.sh -c 1 vm1 >> virtio disk device file "./diskdev" does not exist. >> Creating it ... >> Launching virtual machine "vm1" ... >> vm_setup_memory: Inappropriate ioctl for device >> > > Your userspace is most likely out of sync with the kernel. Could you try > to re-run after a buildworld and installworld? > > best > Neel > > >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to " >> freebsd-virtualization-unsubscribe@freebsd.org" >> > > From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 03:38:17 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 014D9E34 for ; Fri, 27 Sep 2013 03:38:16 +0000 (UTC) (envelope-from dnixon-fnre@nyclocal.net) Received: from homiemail-a28.g.dreamhost.com (caiajhbdcbef.dreamhost.com [208.97.132.145]) by mx1.freebsd.org (Postfix) with ESMTP id D900F2E8B for ; Fri, 27 Sep 2013 03:38:16 +0000 (UTC) Received: from homiemail-a28.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a28.g.dreamhost.com (Postfix) with ESMTP id 9FA9E1B4058 for ; Thu, 26 Sep 2013 20:38:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=nyclocal.net; h=message-id :date:subject:from:to:mime-version:content-type: content-transfer-encoding; s=nyclocal.net; bh=bGrzGUrqnYQraYsXTx RGS5YKG04=; b=Ltfbkw9k8/1FIQFcbimSi1WrMmGWcHrK/fpGOPo0oaVpGkdzBr vVwtOnvdvHHCBgpgmxhwzqMVFmBJte4hJl2s+iFxLLwneB/3BW/9GYq3TDneWYKi umWI3dQ0gigg516/2NmMwc/4JmmxbNOd0zlO/tnsr7YzvqAIZdS0IyUgU= Received: from webmail.nyclocal.net (caiajhbihbdd.dreamhost.com [208.97.187.133]) (Authenticated sender: dnixon-fnre@nyclocal.net) by homiemail-a28.g.dreamhost.com (Postfix) with ESMTPA id 841891B4057 for ; Thu, 26 Sep 2013 20:38:15 -0700 (PDT) Received: from 50.14.66.127 (proxying for 50.14.66.127) (SquirrelMail authenticated user dnixon-fnre@nyclocal.net) by webmail.nyclocal.net with HTTP; Thu, 26 Sep 2013 23:38:16 -0400 Message-ID: <91f45184a3243e881d9f89f3dc2246d3.squirrel@webmail.nyclocal.net> Date: Thu, 26 Sep 2013 23:38:16 -0400 Subject: CFT: PetiteCloud 0.1.3 From: "Dee Nixon" To: "FreeBSD virtualization" User-Agent: SquirrelMail/1.4.21 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 03:38:17 -0000 Version 0.1.3 of PetiteCloud (a front end for bhyve) has been released. We would very much appreciate feedback from anyone inclined to test it. PetiteCloud can be downloaded (as a FreeBSD port) from http://petitecloud.org/port-0.1.3.tar.gz . New Features: * All startup initializations (including loading kernel modules and ifconfig's) are now 100% internal. Only one line needs to be added manually to /etc/rc.conf to start petitecloud at boot time.=20 bootloader.conf needs no modification. * Setting screen added (password and bridged interface are the two current settings available) Bug Fixes: * Fixed wrong assignment of tap on startup * Fixed need for double password entry Instructions can be found in the README file. The user interface is still usable only by experienced sysadmins.=20 (For example, you need to know what a "network interface" is.)=20 Future versions of PetiteCloud will hide some of these details, making PetiteCloud more accessible to a wider range of users, and will have better input validation, etc. From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 07:44:35 2013 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 81AA5422 for ; Fri, 27 Sep 2013 07:44:35 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 38E8528A0 for ; Fri, 27 Sep 2013 07:44:34 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-245-177.lns20.per2.internode.on.net [121.45.245.177]) (authenticated bits=0) by vps1.elischer.org (8.14.6/8.14.6) with ESMTP id r8R7iBdB038733 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 27 Sep 2013 00:44:24 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <52453747.6090706@freebsd.org> Date: Fri, 27 Sep 2013 15:44:07 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= Subject: Re: Xen (and others Hypervisors) how do they handle IPIs? References: <5242F9B8.9090300@freebsd.org> <5242FB81.4090002@citrix.com> <5243927C.9080609@freebsd.org> <5244036D.7060004@citrix.com> In-Reply-To: <5244036D.7060004@citrix.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 07:44:35 -0000 On 9/26/13 5:50 PM, Roger Pau Monné wrote: > On 26/09/13 03:48, Julian Elischer wrote: >> On 9/25/13 11:04 PM, Roger Pau Monné wrote: >>> On 25/09/13 16:56, Julian Elischer wrote: >>>> If CPUs are mapped around, how are IPIs handled? I assume they must be >>>> emulated? >>>> >>>> I've noticed that under Xen (on both Amazon EC2 and a Redhat server) >>>> whenever you schedule a thread it always sits on the run queue for 20 >>>> uSecs before it starts running. It looks to me like it's the IPI taking >>>> a long time to be emulated. >>> This has been improved on the FreeBSD Xen PVHVM port by using PV IPIs >>> instead of the emulated ones, see r255331. It should be faster than the >>> previous emulated implementation. >>> >> I missed that.. thanks! >> Do you (or anyone else) know if this can be used on Amazon EC2? >> And do you need a specific version/configuration of Xen to be able to >> use it? I ran a new GENERIC kernel on an Amazon Xen 4.2 system and it performed a bit better. delays in the running of threads reduced from a constant 20uSecs to a variable amount between 1.5uSecs and 9uSecs. It's still very slow when compared with real hardware. (timing viewed using Schedgraph) Is there a way to see whether PV IPIs are active? (it looks it from the results but that could be so many other things too). Julian > The PV IPIs require Xen version 4.0 or greater, and the PV timer > requires 4.0.1 or greater if I'm not mistaken. > > If you are lucky to get an Amazon instance that uses this Xen version > (or any superior one) they will be activated by default, if not FreeBSD > will switch to the old event delivery method and PV IPIs and PV timer > will be disabled in favour of the emulated ones. > > I guess it's just a matter of time before Amazon switches all their > servers to Xen 4.x. > > Roger. > > > From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 09:17:17 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5DEBF1B6; Fri, 27 Sep 2013 09:17:17 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 633AB2D9F; Fri, 27 Sep 2013 09:17:15 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.90,991,1371081600"; d="scan'208";a="57983762" Received: from accessns.citrite.net (HELO FTLPEX01CL03.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 27 Sep 2013 09:17:08 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.80) with Microsoft SMTP Server id 14.2.342.4; Fri, 27 Sep 2013 05:17:08 -0400 Message-ID: <52454D15.8020407@citrix.com> Date: Fri, 27 Sep 2013 11:17:09 +0200 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Julian Elischer Subject: Re: Xen (and others Hypervisors) how do they handle IPIs? References: <5242F9B8.9090300@freebsd.org> <5242FB81.4090002@citrix.com> <5243927C.9080609@freebsd.org> <5244036D.7060004@citrix.com> <52453747.6090706@freebsd.org> In-Reply-To: <52453747.6090706@freebsd.org> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-DLP: MIA2 Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 09:17:17 -0000 On 27/09/13 09:44, Julian Elischer wrote: > On 9/26/13 5:50 PM, Roger Pau Monné wrote: >> On 26/09/13 03:48, Julian Elischer wrote: >>> On 9/25/13 11:04 PM, Roger Pau Monné wrote: >>>> On 25/09/13 16:56, Julian Elischer wrote: >>>>> If CPUs are mapped around, how are IPIs handled? I assume they must be >>>>> emulated? >>>>> >>>>> I've noticed that under Xen (on both Amazon EC2 and a Redhat server) >>>>> whenever you schedule a thread it always sits on the run queue for 20 >>>>> uSecs before it starts running. It looks to me like it's the IPI >>>>> taking >>>>> a long time to be emulated. >>>> This has been improved on the FreeBSD Xen PVHVM port by using PV IPIs >>>> instead of the emulated ones, see r255331. It should be faster than the >>>> previous emulated implementation. >>>> >>> I missed that.. thanks! >>> Do you (or anyone else) know if this can be used on Amazon EC2? >>> And do you need a specific version/configuration of Xen to be able to >>> use it? > > I ran a new GENERIC kernel on an Amazon Xen 4.2 system and it performed > a bit better. > delays in the running of threads reduced from a constant 20uSecs to a > variable amount between 1.5uSecs and 9uSecs. > It's still very slow when compared with real hardware. > (timing viewed using Schedgraph) > > Is there a way to see whether PV IPIs are active? > (it looks it from the results but that could be so many other things too). The easiest way to see if PV IPIs are active is to run vmstat -i inside of the guest and search for the following lines: irq836: cpu6:r 2 0 irq839: cpu6:irg 2530 70 irq841: cpu6:b 2175 60 This is the number of the different kinds of PV IPIs received on each CPU. You can find the correlation between this names and the IPIs on top of the file x86/xen/hvm.c. Roger. From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 14:50:59 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 77027C20; Fri, 27 Sep 2013 14:50:59 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3737026EE; Fri, 27 Sep 2013 14:50:57 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.90,993,1371081600"; d="scan'208";a="55549937" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 27 Sep 2013 14:50:49 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.2.342.4; Fri, 27 Sep 2013 10:50:49 -0400 Message-ID: <52459B47.8030101@citrix.com> Date: Fri, 27 Sep 2013 16:50:47 +0200 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Shanker Balan Subject: Re: Latest current -CURRENT (rev 255904) panics with "device hyperv" on XenServer 6.2 References: <7943FB36-C6F4-4529-A805-2C467192231F@shankerbalan.net> In-Reply-To: <7943FB36-C6F4-4529-A805-2C467192231F@shankerbalan.net> Content-Type: multipart/mixed; boundary="------------020200080305020009060200" X-DLP: MIA2 Cc: "freebsd-xen@freebsd.org" , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 14:50:59 -0000 --------------020200080305020009060200 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit On 27/09/13 08:16, Shanker Balan wrote: > Helo, > > Now that XENHVM has been merged into the GENERIC kernel, I recompiled todays -CURRENT (rev 255904) > and hit the following panic. It seeems to be HyperV related. > > GENERIC without "device hyperv" works properly. The hypervisor is XenServer 6.2. > > # HyperV drivers > device hyperv # HyperV drivers > > Panic screenshots with backtrace at http://imgur.com/cZsDsKE&VCkh4VS Hello, This is because XenServer enables viridian by default when running HVM guests. That makes Xen also announce itself as HyperV for compatibility reasons, as a workaround you can try to disable viridian support (not sure if this is possible on XenServer). I'm attaching a patch that should solve the problem, Ccing the virt mailing list and the persons that I think are involved in the HyperV support for FreeBSD. Roger. --------------020200080305020009060200 Content-Type: text/plain; charset="UTF-8"; x-mac-type=0; x-mac-creator=0; name="xen_hyperv.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="xen_hyperv.patch" diff --git a/sys/dev/hyperv/vmbus/hv_hv.c b/sys/dev/hyperv/vmbus/hv_hv.c index e3f3ae0..64b10fb 100644 --- a/sys/dev/hyperv/vmbus/hv_hv.c +++ b/sys/dev/hyperv/vmbus/hv_hv.c @@ -88,6 +88,16 @@ hv_vmbus_query_hypervisor_presence(void) { u_int regs[4]; int hyper_v_detected = 0; + + /* + * The Xen Hypervisor also announces itself as HyperV when + * viridian support is enabled, but we should only use Xen + * in this case, so check for Xen first and disable HyperV + * support if Xen is found. + */ + if (vm_guest == VM_GUEST_XEN) + return 0; + do_cpuid(1, regs); if (regs[2] & 0x80000000) { /* if(a hypervisor is detected) */ /* make sure this really is Hyper-V */ diff --git a/sys/x86/xen/hvm.c b/sys/x86/xen/hvm.c index 2286cf0..9539dd1 100644 --- a/sys/x86/xen/hvm.c +++ b/sys/x86/xen/hvm.c @@ -699,6 +699,7 @@ xen_hvm_init(enum xen_hvm_init_type init_type) return; setup_xen_features(); + vm_guest = VM_GUEST_XEN; break; case XEN_HVM_INIT_RESUME: if (error != 0) --------------020200080305020009060200-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 16:30:05 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AEE22B90; Fri, 27 Sep 2013 16:30:05 +0000 (UTC) (envelope-from mail@shankerbalan.net) Received: from 3r6s4.syminet.com (3r6s4.syminet.com [74.80.234.42]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8E5632C47; Fri, 27 Sep 2013 16:30:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=shankerbalan.net; s=x; h=To:References:Message-Id:Content-Transfer-Encoding:Cc:Date:In-Reply-To:From:Subject:Mime-Version:Content-Type; bh=fDIzDw0OqmLSaCpp8zbkYY+mPQ/7Pd3QfJwon+jBj4s=; b=rIEPlL8klsZ7vMPBEsTPvR7Rtsxmrqffv+KMHNT3uKHjQyzk5cmd1C5lhWNvM94tGtky2cpyiKtbzcHSG42Qh0V1dNB8AAoq64A9rN2Es1CEfc+ll4ybKrMk18YLM740; Received: from [103.5.132.53] (helo=buffy.local) by 3r6s4.syminet.com with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1VPavb-0008Oz-Lc; Fri, 27 Sep 2013 09:30:04 -0700 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: Latest current -CURRENT (rev 255904) panics with "device hyperv" on XenServer 6.2 From: Shanker Balan In-Reply-To: <52459E9A.5060801@citrix.com> Date: Fri, 27 Sep 2013 21:59:56 +0530 Content-Transfer-Encoding: quoted-printable Message-Id: <0BDA4CB1-032A-4893-8FB6-14C94FD36D30@shankerbalan.net> References: <7943FB36-C6F4-4529-A805-2C467192231F@shankerbalan.net> <52459B47.8030101@citrix.com> <52459E9A.5060801@citrix.com> To: "freebsd-xen@freebsd.org" X-Mailer: Apple Mail (2.1510) X-Antiabuse: This header was added to track abuse, please include it with any abuse report X-Antiabuse: Primary Hostname - 3r6s4.syminet.com X-Antiabuse: Original Domain - freebsd.org X-Antiabuse: Originator/Caller UID/GID - [105 113] / [105 113] X-Antiabuse: Sender Address Domain - shankerbalan.net Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 16:30:05 -0000 (Doing a proper reply all now) On 27-Sep-2013, at 8:34 PM, Roger Pau Monn=E9 = wrote: > On 27/09/13 16:59, Shanker Balan wrote: >> Roger, >>=20 >> Thank you very much for looking into the issue. >>=20 >> Can I have a SVN version of the patch? "patch" does not seem to like >> the diff (or maybe I am using it incorrectly) >>=20 >> root@fxen1:/usr/src # patch -n < ~/xen_hyperv.patch >> Hmm... I can't seem to find a patch in there anywhere. >=20 > $ patch -p1 < /path/to/patch >=20 > should work AFAIK, this is a git generated diff. If it still doesn't > work applying it by hand shouldn't be that hard. >=20 >=20 Patch works for me on XenServer 6.2. root@fxen1:~ # uname -a FreeBSD fxen1.lab 10.0-ALPHA3 FreeBSD 10.0-ALPHA3 #1 r255912M: Fri Sep = 27 15:27:19 IST 2013 root@fxen1.lab:/usr/obj/usr/src/sys/MYKERNEL = amd64 root@fxen1:~ #=20 root@fxen1:~ # sysctl -a | grep ^device | grep "xen\|hyper" device hyperv device xenpci Thank you very much. Regards. @shankerbalan From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 22:08:00 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C1382990 for ; Fri, 27 Sep 2013 22:08:00 +0000 (UTC) (envelope-from miguelmclara@gmail.com) Received: from mail-we0-x233.google.com (mail-we0-x233.google.com [IPv6:2a00:1450:400c:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5DC692DC7 for ; Fri, 27 Sep 2013 22:08:00 +0000 (UTC) Received: by mail-we0-f179.google.com with SMTP id x55so3257544wes.38 for ; Fri, 27 Sep 2013 15:07:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:cc:content-type; bh=YvrvXz9/JDboNx1epzApXed0I++0T7Iko6BdR+wRFbc=; b=OuSAocfimFLnyMlGWhAbQ0B4Q67S9aDsu6ct1hdHMUml85PYXZgI44RvqKqP7VJqhO rZknM+LH65DWTOBJ/UDIjJo36/Lej/MfWbFZ5pN1KJdu31l5PZ2SlYksIxbd458HvJUL zWswmqg7rGPBOb8JwLf/kruZAMRrb7tMITd73iGN4rWRfWK2ECmWgk3+49wKdH2kbtRe 9H5OcRxF5pnKGp6v5UIuTW54B8xq8nEsij+pGOhB/ZdyN+YCo1BmH+pLr3JQnncKrrZI +4f/xZoMnNJVIhhg1rUq53pTG+bEuUHEovr9XZD9kYePzY67Lf8+rpnt6YixB833xQNr uWqQ== X-Received: by 10.180.189.9 with SMTP id ge9mr4386530wic.52.1380319678752; Fri, 27 Sep 2013 15:07:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.147.68 with HTTP; Fri, 27 Sep 2013 15:07:38 -0700 (PDT) From: Miguel Clara Date: Fri, 27 Sep 2013 22:07:38 +0000 Message-ID: Subject: [SOLVED] Re: bhyve vmrum vm_setup_memory: Inappropriate ioctl for device To: Neel Natu Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 22:08:00 -0000 This was indeed the problem! Solved now. Btw, I guesss FreeBSD is the only guest type supported so far, are there plans for linux/windows? Whats the best place to get updates on this? On Fri, Sep 27, 2013 at 2:35 AM, Miguel Clara wrote: > I was actually just thinking about that... make buildworld is underway... > > I'll post any updates after the userspace install. > > Thanks > > > On Fri, Sep 27, 2013 at 2:31 AM, Neel Natu wrote: > >> Hi Miguel, >> >> On Thu, Sep 26, 2013 at 6:59 PM, Miguel Clara wrote: >> >>> Just wanted to start testing with bhyve, but I'm getting this error >>> creating an exmaple VM: >>> >>> >>> >>> # ./vmrun.sh -c 1 vm1 >>> virtio disk device file "./diskdev" does not exist. >>> Creating it ... >>> Launching virtual machine "vm1" ... >>> vm_setup_memory: Inappropriate ioctl for device >>> >> >> Your userspace is most likely out of sync with the kernel. Could you try >> to re-run after a buildworld and installworld? >> >> best >> Neel >> >> >>> _______________________________________________ >>> freebsd-virtualization@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>> To unsubscribe, send any mail to " >>> freebsd-virtualization-unsubscribe@freebsd.org" >>> >> >> > From owner-freebsd-virtualization@FreeBSD.ORG Sat Sep 28 12:23:53 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D7EA63B8 for ; Sat, 28 Sep 2013 12:23:53 +0000 (UTC) (envelope-from symbolics@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 71C1E296A for ; Sat, 28 Sep 2013 12:23:53 +0000 (UTC) Received: from lemon ([80.7.17.14]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0LjaEi-1W19Bp2Vo0-00bdF5 for ; Sat, 28 Sep 2013 14:23:45 +0200 Received: by lemon (Postfix, from userid 1001) id 7A581EB25B; Sat, 28 Sep 2013 13:07:50 +0100 (BST) Date: Sat, 28 Sep 2013 13:07:50 +0100 From: symbolics@gmx.com To: virtualization@freebsd.org Subject: Booting Linux VMs under Bhyve with fresh -CURRENT Message-ID: <20130928120750.GA48256@lemon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Provags-ID: V03:K0:/UE4jFwHd0y5juQz39CqX1CKGFZUww+OCGVNsSOsu4IvujuCpHV Z1yHOEPC3kgHnQX9b7Zfyao0wNLJdhtUDPGFEqty2jg+zeHrEGv7w5jzQfYCJd9+b6p90pS 2hAd98JIBIPqxUK5uwmZVyEXmEPv+/jm76m3Y+9MuHpBVGYm7NEToeuKqZEpux5NzwHvYbH KRqfc0fR3oYzb7vclEz7A== X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 12:23:53 -0000 Hi I noticed Peter Grehan's commit today that suggests it is now possible to boot Ubuntu Linux under bhyve. I've tried a few things but I can't seem to get it to start. The manual pages for bhyve(8) and bhyvectl(8) seem to be missing at the moment, so I'm not sure how to proceed. From reading vmrun.sh I see it uses bhyveload which is only capable of booting FreeBSD. Experiments with using various bhyve command lines similar to that in vmrun.sh didn't succeed. # bhyve -AI -H -P -c 1 -m 512 \ -s 0:0,hostbridge \ -s 1:0,virtio-blk,ubuntu-01.vm \ -s 2:0,virtio-blk,ubuntu-13.04-server-amd64.iso \ ubuntu vm exit[0] reason VMX rip 0x0000000000000000 inst_length 0 error 0 exit_reason 33 qualification 0x0000000000000000 So, is it actually possibe to run Linux VMs today, given the userland tools in -CURRENT? If so, how is it done? Thanks! --sym From owner-freebsd-virtualization@FreeBSD.ORG Sat Sep 28 12:29:45 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8734947E for ; Sat, 28 Sep 2013 12:29:45 +0000 (UTC) (envelope-from symbolics@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 116242992 for ; Sat, 28 Sep 2013 12:29:45 +0000 (UTC) Received: from lemon ([80.7.17.14]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0LaGJa-1W8KkA3pjK-00m7RR for ; Sat, 28 Sep 2013 14:29:36 +0200 Received: by lemon (Postfix, from userid 1001) id 5AFF5EB250; Sat, 28 Sep 2013 13:29:36 +0100 (BST) Date: Sat, 28 Sep 2013 13:29:36 +0100 From: symbolics@gmx.com To: virtualization@freebsd.org Subject: Booting Linux VMs under Bhyve with fresh -CURRENT Message-ID: <20130928122936.GA48691@lemon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Provags-ID: V03:K0:y5W09cTSK8am68FiyGCUtwsztm9jbq4jpjo6aVtCnFS9WToTPlh gvmavCjP81LTyf2LyPGJ9p50sxvEo8Lagec8VdWqB9xnU7Fx+kxjhIhg5Hy2i5Nvw04iZ1Q dWaulCRd+ORsaeWU/M0V1uZ75Iz3p0/Iw33oIEjgQqbHT61qhP9kXiiMwYAJt65EdH1xGuY 1p1eGMCiokht+SaZCV/HA== X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 12:29:45 -0000 Hi I noticed Peter Grehan's commit today that suggests it is now possible to boot Ubuntu Linux under bhyve. I've tried a few things but I can't seem to get it to start. The manual pages for bhyve(8) and bhyvectl(8) seem to be missing at the moment, so I'm not sure how to proceed. From reading vmrun.sh I see it uses bhyveload which is only capable of booting FreeBSD. Experiments with using various bhyve command lines similar to that in vmrun.sh didn't succeed. # bhyve -AI -H -P -c 1 -m 512 \ -s 0:0,hostbridge \ -s 1:0,virtio-blk,ubuntu-01.vm \ -s 2:0,virtio-blk,ubuntu-13.04-server-amd64.iso \ ubuntu vm exit[0] reason VMX rip 0x0000000000000000 inst_length 0 error 0 exit_reason 33 qualification 0x0000000000000000 So, is it actually possibe to run Linux VMs today, given the userland tools in -CURRENT? If so, how is it done? Thanks! --sym From owner-freebsd-virtualization@FreeBSD.ORG Sat Sep 28 12:34:53 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 924325E6 for ; Sat, 28 Sep 2013 12:34:53 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 550E429D9 for ; Sat, 28 Sep 2013 12:34:52 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 18691121CE; Sat, 28 Sep 2013 22:34:51 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([62.173.8.174]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BOX15664 (AUTH peterg@ptree32.com.au); Sat, 28 Sep 2013 22:34:48 +1000 Message-ID: <5246CCE0.1050801@freebsd.org> Date: Sat, 28 Sep 2013 14:34:40 +0200 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: symbolics@gmx.com Subject: Re: Booting Linux VMs under Bhyve with fresh -CURRENT References: <20130928120750.GA48256@lemon> In-Reply-To: <20130928120750.GA48256@lemon> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 12:34:53 -0000 Hi sym, > So, is it actually possibe to run Linux VMs today, given the userland > tools in -CURRENT? Not quite - I have yet to commit the AHCI emulation that came out of this year's GSoC project to allow an install from ISO. Also, bhyveload isn't used for Linux - I have a modified version of grub2's grub-emu target. I'm putting together a port for that. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sat Sep 28 20:47:35 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D92509D0 for ; Sat, 28 Sep 2013 20:47:35 +0000 (UTC) (envelope-from editor@callfortesting.org) Received: from mail-pd0-f178.google.com (mail-pd0-f178.google.com [209.85.192.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AE50021D6 for ; Sat, 28 Sep 2013 20:47:35 +0000 (UTC) Received: by mail-pd0-f178.google.com with SMTP id w10so4020845pde.9 for ; Sat, 28 Sep 2013 13:47:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=tIObCJcc3sFv4lltcFJolXx7wd5LlPWMiLQvJJCpl98=; b=JrL/GjtImCoXaMCm4OwhXcATcJroQDteLYjtayMGZ7XR9gNMwbs/A6TrWJi5W34mxF ZKvA4YLBa2G9IdLDtYH8h9BPqRr1JRK6SRLxaYVLjlit+hnhSJ3y6BwO3tgL51w3/AXD CljJMsxg2CTmFpusRl58b7PGOXFImQSpuFaOdr9IUTjwaDTAsUcUKKmPeOPvZWklWT2h w0cqHo7oeohP+2StmNwfjvovY+FRLjgR/CUqpxgucaN42RG1/5pH+Gp0XJTFVRNTf24p tPz3NlHQKLLkBUgAy6uwFkpa5i9ZmM3+0lD6mBqIelhqXXfo/ofAeE0sxMVbs/GI1sxt u3bA== X-Gm-Message-State: ALoCoQmjnrPJGkIp/2SYA/jZ+pa2Qs6JVy8vINO9B/m6CVSM/TEoBNTK1J5zse77CMn9g6sly7Kc X-Received: by 10.66.196.168 with SMTP id in8mr18798991pac.18.1380400867998; Sat, 28 Sep 2013 13:41:07 -0700 (PDT) Received: from Michaels-MacBook-Pro.local (c-98-246-202-204.hsd1.or.comcast.net. [98.246.202.204]) by mx.google.com with ESMTPSA id yo2sm21731874pab.8.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 28 Sep 2013 13:41:07 -0700 (PDT) Message-ID: <52473EE5.1040501@callfortesting.org> Date: Sat, 28 Sep 2013 13:41:09 -0700 From: Michael Dexter User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Miguel Clara Subject: Re: [SOLVED] Re: bhyve vmrum vm_setup_memory: Inappropriate ioctl for device References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 20:47:35 -0000 On 9/27/13 3:07 PM, Miguel Clara wrote: > Btw, I guesss FreeBSD is the only guest type supported so far, are there > plans for linux/windows? Whats the best place to get updates on this? Experimental Linux and OpenBSD soon. Windows to be determined. Michael