From owner-freebsd-virtualization@freebsd.org Sun Aug 26 00:47:38 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75047109AF3F for ; Sun, 26 Aug 2018 00:47:38 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: from mail-qt0-x234.google.com (mail-qt0-x234.google.com [IPv6:2607:f8b0:400d:c0d::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C6F588962 for ; Sun, 26 Aug 2018 00:47:38 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: by mail-qt0-x234.google.com with SMTP id f19-v6so12844791qtf.2 for ; Sat, 25 Aug 2018 17:47:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rtuGUzi0XHTZacNkd7BpIyucWF0YN31ToO4NTIVjOmo=; b=hPQUX0InR91RFSZnir+Elsxrp/ztV858pjR7ctoH9p1jENayMFMP6zVOaxThoBuBlj aHSAv41ky7WXnmYsBIDPAybnDBNKf75O/0PZ0FinSxwDUT9YcgKI/02ijI4NkmobA/i7 EMZ5X4tjKDvhOguEVVOD2vaqprPuc9Rs66HlMOCRRSWk3htjdwZjoBJsQ+33SYVbfyiP mEreNV9X7FZqUtqJVacSscUSvxNWN2WmO5eYJpJeQnJraOeviEb045eyxOd4/HOgvWS0 IpZYSONYMyAfPfzzBE1jhk2AJy9j98eSLzrngoiLkD8z8X2m3Wi87JC8iKguvAocdzdf 1/6A== X-Gm-Message-State: APzg51Cj3ilJEO+EadE1zEk3dwqhPbEkhtWpbjlAlMVXc5Ssp5p+uW19 UtYPsBJLWX1IgwVSRt4BDONxCOkzAnyQ0k7i5t7MCK6JI5A= X-Google-Smtp-Source: ANB0VdaNMn7vNKIFip0+3ZD2l2cp1Z5kj707WBg50+GNnHuQZ6u/itsaqshv3DyZG6HfvYuYeUIGyGisUZjx/sYBERE= X-Received: by 2002:a0c:e64a:: with SMTP id c10-v6mr7771588qvn.199.1535244457548; Sat, 25 Aug 2018 17:47:37 -0700 (PDT) MIME-Version: 1.0 References: <6699362.kGnQbBhLUH@linux-9daj> In-Reply-To: <6699362.kGnQbBhLUH@linux-9daj> From: Paul Webster Date: Sun, 26 Aug 2018 01:47:26 +0100 Message-ID: Subject: Re: Query regarding tutorials (Please have a quick read of me!) To: vixie@fsi.io Cc: "freebsd-virtua." Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 00:47:38 -0000 Just making sure 2016 works with no issues: https://1drv.ms/u/s!AkvwG6Qrk79x3BvNWJt6O-ew88iM looks all shiny and good to go for tomorrow :) On Sat, 25 Aug 2018 at 18:45, Paul Vixie wrote: > On Saturday, August 25, 2018 8:55:05 AM UTC Paul Webster via freebsd- > virtualization wrote: > > ... I personally have: > > > > Windows Server 2012 > > CentOS 6 > > Debian > > FreeBSD > > OpenBSD > > and a few other more strange linuxs I believe (got a zpool full of the > > zvols (not named very well ;))) ... > > i have no exotic linuxes, but two fairly modern debians. i remain > mystified by > the behaviour of grub-bhyve, which usually fails (no meaningful error > message) > and seems like won't run from /etc/rc.local (must be run from a shell.) > so, > one of my bhyve servers can't restart automatically, and i don't know why. > and > i recently saw that the grub-bhyve package now lacks a maintainer, and may > be > removed. > > what this means is, a video on running linux in a bhyve would be > appreciated. > > > anyhow IRC tells me that a lot of users > > have trouble getting windows server working. I know obviously that 2012 > > works but I am willing to take a swing at 2016 if people would want that. > > > > Or would the community benefit more of an example of 2012 (which is much > > lighter)? > > > > Or perhaps something entirely different, any replies would be more than > > welcome :) > > because of the internal container boundaries of windows 10 and server > 2016, i > won't run anything older. so if you're doing a windows server on bhyve > video, > i would want it to be 2016. > > -- > Vixie > > From owner-freebsd-virtualization@freebsd.org Sun Aug 26 00:48:29 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EC133109AFA1 for ; Sun, 26 Aug 2018 00:48:28 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: from mail-qk0-x22a.google.com (mail-qk0-x22a.google.com [IPv6:2607:f8b0:400d:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A40088A3D for ; Sun, 26 Aug 2018 00:48:28 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: by mail-qk0-x22a.google.com with SMTP id f17-v6so8263787qkh.4 for ; Sat, 25 Aug 2018 17:48:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=O5hNXRRb9NCzAHTFG7r72XTE+DBJCuIvIT8BucuntlE=; b=gE8DWhXU0hehd1eVpIfhxRd25ORE56VDC7dGVmVZvkX2ERDVNqPoMU40Fk0Ehb/xQV E1Z4rThHbgTn2Lz+DBf2y9tEQwm0LxpyNOBTgJdlfJFe2CsSm+ohoLnhw4e7Ql/2iNzv zQu2c7b314DhAoPchxUooDkGtybCF43BQFiyH+RcV3r5rIkBM1ZMWJLHKytTneHJvlqk 9gfnzuySN64xCQS1pu1D3pLFFR5B0PbahP1REVxiy+tWjHklpr+Mk/qc6abQ1nOJ9/sa dhMDBKhL17iTYpnLxTjI7dppTJ6B0Y9L4UwAxcxUGE2JUdVCiUQ6mpsKhD/GMoiABgqM tbFA== X-Gm-Message-State: APzg51CWPzMBH8ne+WtD3T7uMEi9isjt68m/b9v6zlw6llzVoeEbR6oa IKHBkA2w/Nmm0l7uT29h3C/Z4+jC2jQCKp3UBG2YzEg5 X-Google-Smtp-Source: ANB0VdauMn4ZPwJl1mSNe8QjXh18gDs+ghGXDnwUMv2oiWC1tJSvLqQicNh9/eq4kcdntwa8AAlTRQDQbocb36aeijE= X-Received: by 2002:ae9:c30f:: with SMTP id n15-v6mr7636069qkg.88.1535244508189; Sat, 25 Aug 2018 17:48:28 -0700 (PDT) MIME-Version: 1.0 References: <20180825184929.A069E156E400@mail.bitblocks.com> In-Reply-To: <20180825184929.A069E156E400@mail.bitblocks.com> From: Paul Webster Date: Sun, 26 Aug 2018 01:48:17 +0100 Message-ID: Subject: Re: Query regarding tutorials (Please have a quick read of me!) To: bakul@bitblocks.com, "freebsd-virtua." Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 00:48:29 -0000 plan9 yikes! I have never tried to set that up ever in the first place, I will give it a shot though if not just for the fun though! :) On Sat, 25 Aug 2018 at 19:49, Bakul Shah wrote: > On Sat, 25 Aug 2018 09:55:05 +0100 Paul Webster via freebsd-virtualization > wrote: > > Hey all, > > > > A while ago I created a youtube channel to start doing video example > > versions of the FreeBSD documentation, I have decided that this weekend I > > would take a swing at showing howto setup a Bhyve VM, I personally have: > > > ... > > > > Or perhaps something entirely different, any replies would be more than > > welcome :) > > I have 9front (plan9) running. A video on how to set that up > would be a worthy addition and something entirely different! > From owner-freebsd-virtualization@freebsd.org Sun Aug 26 02:46:51 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5832109F402 for ; Sun, 26 Aug 2018 02:46:50 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A8D98CEC4 for ; Sun, 26 Aug 2018 02:46:50 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 9E1881E1BE for ; Sun, 26 Aug 2018 02:46:49 +0000 (UTC) To: freebsd-virtualization@freebsd.org References: <6699362.kGnQbBhLUH@linux-9daj> From: Allan Jude Openpgp: preference=signencrypt Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Subject: Re: Query regarding tutorials (Please have a quick read of me!) Message-ID: Date: Sat, 25 Aug 2018 22:46:42 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <6699362.kGnQbBhLUH@linux-9daj> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="F5nYKphqCMJPsZ40H6BViS1tw4TUM9U9c" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 02:46:51 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --F5nYKphqCMJPsZ40H6BViS1tw4TUM9U9c Content-Type: multipart/mixed; boundary="Wa9ek5u47Z9OROT5aw607gSfnqTUlJUv8"; protected-headers="v1" From: Allan Jude To: freebsd-virtualization@freebsd.org Message-ID: Subject: Re: Query regarding tutorials (Please have a quick read of me!) References: <6699362.kGnQbBhLUH@linux-9daj> In-Reply-To: <6699362.kGnQbBhLUH@linux-9daj> --Wa9ek5u47Z9OROT5aw607gSfnqTUlJUv8 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2018-08-25 13:45, Paul Vixie wrote: > On Saturday, August 25, 2018 8:55:05 AM UTC Paul Webster via freebsd- > virtualization wrote: >> ... I personally have: >> >> Windows Server 2012 >> CentOS 6 >> Debian >> FreeBSD >> OpenBSD >> and a few other more strange linuxs I believe (got a zpool full of the= >> zvols (not named very well ;))) ... >=20 > i have no exotic linuxes, but two fairly modern debians. i remain mysti= fied by=20 > the behaviour of grub-bhyve, which usually fails (no meaningful error m= essage)=20 > and seems like won't run from /etc/rc.local (must be run from a shell.)= so,=20 > one of my bhyve servers can't restart automatically, and i don't know w= hy. and=20 What does the invocation look like? Does this work: printf "\n" | grub-bhyve ...your-stuff-here... As an alternative, you can now use the UEFI_CSM firmware for bhyve, and it'll boot from the boot blocks inside the VM, like xen/kvm/qemu/virtualb= ox > i recently saw that the grub-bhyve package now lacks a maintainer, and = may be=20 > removed. >=20 > what this means is, a video on running linux in a bhyve would be apprec= iated. >=20 >> anyhow IRC tells me that a lot of users >> have trouble getting windows server working. I know obviously that 201= 2 >> works but I am willing to take a swing at 2016 if people would want th= at. >> >> Or would the community benefit more of an example of 2012 (which is mu= ch >> lighter)? >> >> Or perhaps something entirely different, any replies would be more tha= n >> welcome :) >=20 > because of the internal container boundaries of windows 10 and server 2= 016, i=20 > won't run anything older. so if you're doing a windows server on bhyve = video,=20 > i would want it to be 2016. >=20 --=20 Allan Jude --Wa9ek5u47Z9OROT5aw607gSfnqTUlJUv8-- --F5nYKphqCMJPsZ40H6BViS1tw4TUM9U9c Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJbghSZAAoJEBmVNT4SmAt+p2gP/2DUednGq1kRmiXmHZmmdEua fGt/FmY7gU08h2PBZ8oDDe+w1OylSklMXgMsZbpzaA8ntYHfdurOv67KNm9oGq2k k/e/q545DkknOo6Nd2EgIZP0gGPEyn6vUw9S5HBmOLCFyRUsTbJ8JZ+/8eSnv+KA fVL8n1RjT7osYn9KxMcYktbc1VtZ0/y5BfqhZWMzInNdU/4CKFXabkAbGtFlgkJK lKhVvY2XGHmHtTNSdMDhRllr3FDZdTcxbo4TwR+gnSAINdtujNQ6/q7uq/MkQbgF qVYOSU4WeB0RiI6LLlEvsBLKmHdFBEIf0HZiGOo7SMTd004VWOotoZHspctDBfII qX/rtfSKt9WLpOK6QG/pGn513wJHlFc0yoN9kH7O8ywEGc3jBkB2f431PtRmCnA0 YAmfkXgC1JUbc1hT3ZhS9GRcZktfvkKClSi4pPlxb+EdVokuxIkS3veIEcH7kzcx lpqYAErJKSQUik4bfDF2BbSp1f0Q4cU8+3ugT5TjmsGOyt0dGLiSs3rA55cvtCeN aeCJyUbATKLn0p5H6oi8zbzqum27xtm8TtEST9fPqUGCubF8e/puZFXNnlnKh0WU DaYlkjjOnfj4mFDF3SuTahDONMuuDdtwNycPkFRkkXWqb88US7kIlw73zE7V+f1K SvoD78J5EYx8IaQ+Jjit =8ZMD -----END PGP SIGNATURE----- --F5nYKphqCMJPsZ40H6BViS1tw4TUM9U9c-- From owner-freebsd-virtualization@freebsd.org Sun Aug 26 05:55:04 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF80A10120E5 for ; Sun, 26 Aug 2018 05:55:04 +0000 (UTC) (envelope-from vixie@fsi.io) Received: from mail.fsi.io (mail.fsi.io [104.244.13.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 74A1F72725; Sun, 26 Aug 2018 05:55:04 +0000 (UTC) (envelope-from vixie@fsi.io) X-Virus-Scanned: amavisd-new at fsi.io Sender: vixie@fsi.io Received: from linux-9daj.localnet (dhcp-181.access.lah1.vix.su [24.104.150.181]) (Authenticated sender: vixie) by mail.fsi.io (Postfix) with ESMTPSA id 0EB8D609BF; Sun, 26 Aug 2018 05:54:48 +0000 (UTC) From: Paul Vixie To: freebsd-virtualization@freebsd.org Subject: Re: Query regarding tutorials (Please have a quick read of me!) Date: Sun, 26 Aug 2018 05:54:47 +0000 Message-ID: <5449454.ijkHvH96lN@linux-9daj> Organization: Farsight Security, Inc. In-Reply-To: References: <6699362.kGnQbBhLUH@linux-9daj> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 05:55:05 -0000 On Sunday, August 26, 2018 2:46:42 AM UTC Allan Jude wrote: > On 2018-08-25 13:45, Paul Vixie wrote: > > ... i remain > > mystified by the behaviour of grub-bhyve, which usually fails (no > > meaningful error message) and seems like won't run from /etc/rc.local > > (must be run from a shell.) so, one of my bhyve servers can't restart > > automatically, and i don't know why. ... > What does the invocation look like? grub-bhyve -r $defdev -m $vm.map -M $pm -c /dev/$sc $vm \ > ${vm}.output 2>&1 \ || { echo "grub-bhyve failed" >&2; exit 1; } > Does this work: > printf "\n" | grub-bhyve ...your-stuff-here... i'll give it a try. testing this particular path requires a reboot, and the system is otherwise running in production... so give me a day or two. > > As an alternative, you can now use the UEFI_CSM firmware for bhyve, and > it'll boot from the boot blocks inside the VM, like xen/kvm/qemu/virtualbox this sounds far preferable! is there an example or other documentation? -- Vixie From owner-freebsd-virtualization@freebsd.org Sun Aug 26 05:57:30 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE2EC10121D2 for ; Sun, 26 Aug 2018 05:57:29 +0000 (UTC) (envelope-from vixie@fsi.io) Received: from mail.fsi.io (mail.fsi.io [104.244.13.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8B02E727FF; Sun, 26 Aug 2018 05:57:29 +0000 (UTC) (envelope-from vixie@fsi.io) X-Virus-Scanned: amavisd-new at fsi.io Sender: vixie@fsi.io Received: from linux-9daj.localnet (dhcp-181.access.lah1.vix.su [24.104.150.181]) (Authenticated sender: vixie) by mail.fsi.io (Postfix) with ESMTPSA id 720F0608CF; Sun, 26 Aug 2018 05:57:23 +0000 (UTC) From: Paul Vixie To: Paul Vixie Cc: freebsd-virtualization@freebsd.org, Allan Jude Subject: Re: Query regarding tutorials (Please have a quick read of me!) Date: Sun, 26 Aug 2018 05:57:22 +0000 Message-ID: <8898849.4pPJjUbtcD@linux-9daj> Organization: Farsight Security, Inc. In-Reply-To: <5449454.ijkHvH96lN@linux-9daj> References: <5449454.ijkHvH96lN@linux-9daj> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 05:57:30 -0000 On Sunday, August 26, 2018 5:54:47 AM UTC Paul Vixie wrote: > On Sunday, August 26, 2018 2:46:42 AM UTC Allan Jude wrote: > > On 2018-08-25 13:45, Paul Vixie wrote: > > > ... i remain > > > mystified by the behaviour of grub-bhyve, which usually fails (no > > > meaningful error message) and seems like won't run from /etc/rc.local > > > (must be run from a shell.) so, one of my bhyve servers can't restart > > > automatically, and i don't know why. ... > > > > What does the invocation look like? trying again: grub-bhyve -r $defdev -m $vm.map -M $pm -c /dev/$sc $vm \ > ${vm}.output 2>&1 \ || { echo "grub-bhyve failed" >&2; exit 1; } -- Vixie From owner-freebsd-virtualization@freebsd.org Sun Aug 26 06:11:35 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C96F710128F7 for ; Sun, 26 Aug 2018 06:11:35 +0000 (UTC) (envelope-from vixie@fsi.io) Received: from mail.fsi.io (mail.fsi.io [104.244.13.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6E04072E2B for ; Sun, 26 Aug 2018 06:11:35 +0000 (UTC) (envelope-from vixie@fsi.io) X-Virus-Scanned: amavisd-new at fsi.io Sender: vixie@fsi.io Received: from linux-9daj.localnet (dhcp-181.access.lah1.vix.su [24.104.150.181]) (Authenticated sender: vixie) by mail.fsi.io (Postfix) with ESMTPSA id 17725608BC; Sun, 26 Aug 2018 06:10:59 +0000 (UTC) From: Paul Vixie To: freebsd-virtualization@freebsd.org Subject: Re: Query regarding tutorials (Please have a quick read of me!) Date: Sun, 26 Aug 2018 06:10:58 +0000 Message-ID: <6508874.anr9O9xsbq@linux-9daj> Organization: Farsight Security, Inc. In-Reply-To: <8898849.4pPJjUbtcD@linux-9daj> References: <5449454.ijkHvH96lN@linux-9daj> <8898849.4pPJjUbtcD@linux-9daj> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7Bit X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 06:11:36 -0000 On Sunday, August 26, 2018 5:57:22 AM UTC Paul Vixie wrote: > On Sunday, August 26, 2018 5:54:47 AM UTC Paul Vixie wrote: > > On Sunday, August 26, 2018 2:46:42 AM UTC Allan Jude wrote: > > > What does the invocation look like? > > trying again: > > ... feh. see attached. -- Vixie From owner-freebsd-virtualization@freebsd.org Sun Aug 26 16:25:25 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1440D107A768 for ; Sun, 26 Aug 2018 16:25:25 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: from mail-qk0-x236.google.com (mail-qk0-x236.google.com [IPv6:2607:f8b0:400d:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A66F785C13 for ; Sun, 26 Aug 2018 16:25:24 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: by mail-qk0-x236.google.com with SMTP id z78-v6so8992223qka.0 for ; Sun, 26 Aug 2018 09:25:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=12UY8VhXQWSJcLslJKPlaRdwuC+Irlp1IZ/cqlPs6o0=; b=BqgAd4fHgNvqxIfE2Pu/In4CHHHAot3qyMcD7r1f1oN8bD+Osko5ssE6p6dFSUovuc Z8+Zy9VpgLnUpywtH1ROm/PKCSmSuFM9o4lNYRFlV/RX2fYQCUTrbz8S3ymrQF2lHUuP 8Kjt7WrUMwkh3dAWjUoO8GVAFZrt9iKkaXCv3ID5dj2tavY0+FjUuSwmQdwaqfSuCo74 b5yaNyZ5PJp4+rQwzlANEgYe9sTnVS8eOCSU9uXGWlhuSaVbx5dYcyodtX5FZ9EYuV2B LWFiDXlaQtzc2bnHp8blEftMv7xkdyO4fOgH7gYPtQNB0jS5oVBhRpLn6HtdcfJ15dA6 ITDg== X-Gm-Message-State: APzg51AtljijDJ6brZ3BEyRPD8yNo86sMxoBJM5lWh4rinX6Hgg3/E+2 iN52/THRV1oSbh12ACvqMWPmfWQ17wauBbFN6QI= X-Google-Smtp-Source: ANB0VdZoA0QfWgo5Vlzj1lSfiq8AX/pIwzBs2RLp0g9lkBoR2JcdOUbEEQaKIRTWG6B50cmuuDoFibr65SXVIhzCOes= X-Received: by 2002:a37:bac7:: with SMTP id k190-v6mr9961671qkf.138.1535300724108; Sun, 26 Aug 2018 09:25:24 -0700 (PDT) MIME-Version: 1.0 References: <6699362.kGnQbBhLUH@linux-9daj> In-Reply-To: <6699362.kGnQbBhLUH@linux-9daj> From: Paul Webster Date: Sun, 26 Aug 2018 17:25:13 +0100 Message-ID: Subject: Re: Query regarding tutorials (Please have a quick read of me!) To: Paul Vixie , "freebsd-virtua." Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 16:25:25 -0000 howto start stuff auto: https://www.youtube.com/watch?v=MidLqBs4_B8&feature=youtu.be On Sat, 25 Aug 2018 at 18:45, Paul Vixie wrote: > On Saturday, August 25, 2018 8:55:05 AM UTC Paul Webster via freebsd- > virtualization wrote: > > ... I personally have: > > > > Windows Server 2012 > > CentOS 6 > > Debian > > FreeBSD > > OpenBSD > > and a few other more strange linuxs I believe (got a zpool full of the > > zvols (not named very well ;))) ... > > i have no exotic linuxes, but two fairly modern debians. i remain > mystified by > the behaviour of grub-bhyve, which usually fails (no meaningful error > message) > and seems like won't run from /etc/rc.local (must be run from a shell.) > so, > one of my bhyve servers can't restart automatically, and i don't know why. > and > i recently saw that the grub-bhyve package now lacks a maintainer, and may > be > removed. > > what this means is, a video on running linux in a bhyve would be > appreciated. > > > anyhow IRC tells me that a lot of users > > have trouble getting windows server working. I know obviously that 2012 > > works but I am willing to take a swing at 2016 if people would want that. > > > > Or would the community benefit more of an example of 2012 (which is much > > lighter)? > > > > Or perhaps something entirely different, any replies would be more than > > welcome :) > > because of the internal container boundaries of windows 10 and server > 2016, i > won't run anything older. so if you're doing a windows server on bhyve > video, > i would want it to be 2016. > > -- > Vixie > > From owner-freebsd-virtualization@freebsd.org Sun Aug 26 19:04:16 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4C32107F41B for ; Sun, 26 Aug 2018 19:04:15 +0000 (UTC) (envelope-from vixie@fsi.io) Received: from mail.fsi.io (mail.fsi.io [104.244.13.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8AECD8D780 for ; Sun, 26 Aug 2018 19:04:15 +0000 (UTC) (envelope-from vixie@fsi.io) X-Virus-Scanned: amavisd-new at fsi.io Sender: vixie@fsi.io Received: from linux-9daj.localnet (dhcp-181.access.lah1.vix.su [24.104.150.181]) (Authenticated sender: vixie) by mail.fsi.io (Postfix) with ESMTPSA id E2A8A604F2; Sun, 26 Aug 2018 19:04:09 +0000 (UTC) From: Paul Vixie To: Paul Webster Cc: "freebsd-virtua." Subject: Re: Query regarding tutorials (Please have a quick read of me!) Date: Sun, 26 Aug 2018 19:04:08 +0000 Message-ID: <3626542.UnW4ZVN1pJ@linux-9daj> Organization: Farsight Security, Inc. In-Reply-To: References: <6699362.kGnQbBhLUH@linux-9daj> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 19:04:16 -0000 On Sunday, August 26, 2018 4:25:13 PM UTC Paul Webster wrote: > howto start stuff auto: > https://www.youtube.com/watch?v=MidLqBs4_B8&feature=youtu.be very entertaining. some notes: note that rc.local is executed by the "." command from the /bin/sh instance that runs /etc/rc.d/local, and so, #! isn't relevant there, nor is "chmod +x". net.link.ether.inet.proxyall is documented in arp(4) and just means don't drop arp requests about addresses that aren't local. i prefer to let the bhyve guests answer for themselves, which they'll do, because the arp request is sent to the all ones address, and thus reaches all the guests. i do not use daemon -r for bhyve, because it ignores the the exit status. if bhyve exits with 0 i want to loop (recreate bhyve), if it's nonzero i want it to exit (poweroff, shutdown, or kernel panic). according to bhyve(8): > EXIT STATUS > > Exit status indicates how the VM was terminated: > > 0 rebooted > 1 powered off > 2 halted > 3 triple fault you might be planning to explain how you interpret these exit statuses in the shell script you run from daemon, which in turn starts bhyve. however, it looks as if daemon -r only avoids the restart loop if it receives a SIGTERM, and i don't know how you'll deliver that. my /etc/rc.local script looks like this: > while true; do > > ... > > # we're using file descriptor #3 to get the bhyve exit code out. ugly. > set $( ( ( sh vmrun.sh \ > -c $cpucount \ > -m $memorysize \ > -t tap$tapif \ > -d /dev/zvol/$zvol \ > -C /dev/$cons \ > -x \ > $@ $vm \ > 2>&1; \ > echo $? >&3; ) | logger -t "$tag" ) 3>&1 ) > bhyve_exit=$1 > > echo === $bhyve_exit > if [ "$bhyve_exit" -ne 0 ]; then > break > fi > > sleep 10 > done -- Vixie From owner-freebsd-virtualization@freebsd.org Sun Aug 26 19:56:18 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 022AE10806FF for ; Sun, 26 Aug 2018 19:56:18 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: from mail-qt0-x234.google.com (mail-qt0-x234.google.com [IPv6:2607:f8b0:400d:c0d::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9CFB18EFF7 for ; Sun, 26 Aug 2018 19:56:17 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: by mail-qt0-x234.google.com with SMTP id x7-v6so15803620qtk.5 for ; Sun, 26 Aug 2018 12:56:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0OHzyloSAZ56rJvPMuK5cRrmWgCNR6GTxvXaAEw32iE=; b=q1uUrNI8yD5kih+vJn8iUtGfS9ia0wVtsBUvHAebopO0LutyB+59EfDidThXuczzHa is1Vl7Pt/uf30YSNVGehfWMQ4j1dZbsYN9YrxfVypSplK/aLKbfcTcF56HVH7mNyzP3F +bTLxcnDQmT83PY4zzcYK65dzm0yyllL84Z83OnJUo9kACp6NbZYcdqLrAANfdnC4O5A MoSMaump6cNBdD9Zqf+Y6zh8+1zgjKLf4mV0QR3xJ4UJVfF9ju4bl/D160lT6LqtIsiv C2iJUpRrKG9BCNg7hD1aGeYlAjQ1scfK80DUTf29Cl/EI/KYPfaiTDgUKv7KfzEtFa4R bKJg== X-Gm-Message-State: APzg51A4fUFS0UI+10c04gvU0ijz0qeI8i7Gd9aR33gqX0msmqDg7PpJ fcTfVdqW0jKSz9vstAty59sm8b8saXjOhQs6Ozo= X-Google-Smtp-Source: ANB0VdbnY218m/eQ1L0/wn11F3iO1cjO/okZw7M3pAbLJz+LQ6aHbuELOD+VNBBzh/IiLTTT1NhXOv/KU6xBfWDzZME= X-Received: by 2002:a0c:da8e:: with SMTP id z14-v6mr10626339qvj.171.1535313376040; Sun, 26 Aug 2018 12:56:16 -0700 (PDT) MIME-Version: 1.0 References: <6699362.kGnQbBhLUH@linux-9daj> <3626542.UnW4ZVN1pJ@linux-9daj> In-Reply-To: <3626542.UnW4ZVN1pJ@linux-9daj> From: Paul Webster Date: Sun, 26 Aug 2018 20:56:04 +0100 Message-ID: Subject: Re: Query regarding tutorials (Please have a quick read of me!) To: Paul Vixie Cc: "freebsd-virtua." Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 19:56:18 -0000 indeed I did polish it a bit and upload: https://youtu.be/w0WuoKVWAgI ;) Thank you for the script though! I will have a look at modifying it in the git On Sun, 26 Aug 2018 at 20:04, Paul Vixie wrote: > On Sunday, August 26, 2018 4:25:13 PM UTC Paul Webster wrote: > > howto start stuff auto: > > https://www.youtube.com/watch?v=MidLqBs4_B8&feature=youtu.be > > very entertaining. some notes: > > note that rc.local is executed by the "." command from the /bin/sh > instance > that runs /etc/rc.d/local, and so, #! isn't relevant there, nor is "chmod > +x". > > net.link.ether.inet.proxyall is documented in arp(4) and just means don't > drop > arp requests about addresses that aren't local. i prefer to let the bhyve > guests answer for themselves, which they'll do, because the arp request is > sent to the all ones address, and thus reaches all the guests. > > i do not use daemon -r for bhyve, because it ignores the the exit status. > if > bhyve exits with 0 i want to loop (recreate bhyve), if it's nonzero i want > it > to exit (poweroff, shutdown, or kernel panic). according to bhyve(8): > > > EXIT STATUS > > > > Exit status indicates how the VM was terminated: > > > > 0 rebooted > > 1 powered off > > 2 halted > > 3 triple fault > > you might be planning to explain how you interpret these exit statuses in > the > shell script you run from daemon, which in turn starts bhyve. however, it > looks as if daemon -r only avoids the restart loop if it receives a > SIGTERM, > and i don't know how you'll deliver that. my /etc/rc.local script looks > like > this: > > > while true; do > > > > ... > > > > # we're using file descriptor #3 to get the bhyve exit code out. > ugly. > > set $( ( ( sh vmrun.sh \ > > -c $cpucount \ > > -m $memorysize \ > > -t tap$tapif \ > > -d /dev/zvol/$zvol \ > > -C /dev/$cons \ > > -x \ > > $@ $vm \ > > 2>&1; \ > > echo $? >&3; ) | logger -t "$tag" ) 3>&1 ) > > bhyve_exit=$1 > > > > echo === $bhyve_exit > > if [ "$bhyve_exit" -ne 0 ]; then > > break > > fi > > > > sleep 10 > > done > > -- > Vixie > > From owner-freebsd-virtualization@freebsd.org Sun Aug 26 20:07:52 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B34711080CDA for ; Sun, 26 Aug 2018 20:07:52 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 367848F630 for ; Sun, 26 Aug 2018 20:07:52 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 750121F25B; Sun, 26 Aug 2018 20:07:45 +0000 (UTC) To: Paul Vixie , freebsd-virtualization@freebsd.org References: <6699362.kGnQbBhLUH@linux-9daj> <5449454.ijkHvH96lN@linux-9daj> From: Allan Jude Openpgp: preference=signencrypt Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Subject: Re: Query regarding tutorials (Please have a quick read of me!) Message-ID: Date: Sun, 26 Aug 2018 16:07:41 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <5449454.ijkHvH96lN@linux-9daj> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="42QrO9LWLikv5vVJkGxXtlwTlp6W7702Y" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 20:07:53 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --42QrO9LWLikv5vVJkGxXtlwTlp6W7702Y Content-Type: multipart/mixed; boundary="WTXQXkvVpefn85VKCMaTMdudzA9uDha6R"; protected-headers="v1" From: Allan Jude To: Paul Vixie , freebsd-virtualization@freebsd.org Message-ID: Subject: Re: Query regarding tutorials (Please have a quick read of me!) References: <6699362.kGnQbBhLUH@linux-9daj> <5449454.ijkHvH96lN@linux-9daj> In-Reply-To: <5449454.ijkHvH96lN@linux-9daj> --WTXQXkvVpefn85VKCMaTMdudzA9uDha6R Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2018-08-26 01:54, Paul Vixie wrote: > On Sunday, August 26, 2018 2:46:42 AM UTC Allan Jude wrote: >> On 2018-08-25 13:45, Paul Vixie wrote: >>> ... i remain >>> mystified by the behaviour of grub-bhyve, which usually fails (no >>> meaningful error message) and seems like won't run from /etc/rc.local= >>> (must be run from a shell.) so, one of my bhyve servers can't restart= >>> automatically, and i don't know why. ... >=20 >> What does the invocation look like? >=20 > grub-bhyve -r $defdev -m $vm.map -M $pm -c /dev/$sc $vm \ > > ${vm}.output 2>&1 \ > || { > echo "grub-bhyve failed" >&2; > exit 1; > } >=20 >> Does this work: >> printf "\n" | grub-bhyve ...your-stuff-here... >=20 > i'll give it a try. testing this particular path requires a reboot, and= the=20 > system is otherwise running in production... so give me a day or two. >=20 >> >> As an alternative, you can now use the UEFI_CSM firmware for bhyve, an= d >> it'll boot from the boot blocks inside the VM, like xen/kvm/qemu/virtu= albox >=20 > this sounds far preferable! is there an example or other documentation?= >=20 first, get the firmware images: pkg install bhyve-firmware Then, instead of invoking grub-bhyve, add this extra flag to your bhyve invokation: -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd Or if you want old style BIOS booting: -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CSM.fd --=20 Allan Jude --WTXQXkvVpefn85VKCMaTMdudzA9uDha6R-- --42QrO9LWLikv5vVJkGxXtlwTlp6W7702Y Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJbgwiQAAoJEBmVNT4SmAt+tCEP/0Glym6xvk0Igrqb4iULkXkL YRjH5j0PaZD4qMPKyFKdKdUSpbmSHhFf3crnAtfpG9p5BoKc+NbLOP3ImwT4RgSF KjJrwIqDd50OjPqXra3kk701nM14hE+PYL4yA94FD4SijNACboIneS1vbf9S9O4i FFOh0W9bfHyFCCypFJ1v2DaWQtr13sBvcZ3f1UHOD9L4d7LuvP7mNewcusOtb5S1 qgDOEWDRSn1ILzq3eM6odFIzQ3nIrJ6836IHfxgpMWf02B/hrrjir1Byd213WxyR GeT150bPt/02WSLKTDhqIixv48ViDbrPVJfEEgtp9y4RMb0D0A2UaLWSMzk9IfrJ OjZ4JoH84H2/VnlwDsrxg/tydx4nAWUfGVQfivtyMwpckmJ/Q/DW+UiswyTzhnI+ 1PCkbQxiqjRNhu+7rsneAQcp3XhhPLJg4FZWxf+kc5gG0IaZhUTTCn90wguUYGa/ V+5wFxgvUVwqkDvvXkLDTXa6kSiIiHj28cGQD6RkH3dtU6vA9XKTv6x8xSfxNtLw /EwofFBr3upkPKt2HQijjUWcbXmex0nnngg2CEep8xUfpUOcsc/7e0sfoAF/qDSK W/Z9T2j1tzamqozHJbFvN4Lnm0StMNxK3Q7qlroa6TjFh4FdDaJzkHOUOyM/8nJx RfKkRW1+CPbB9BNLMcB/ =Wbl+ -----END PGP SIGNATURE----- --42QrO9LWLikv5vVJkGxXtlwTlp6W7702Y-- From owner-freebsd-virtualization@freebsd.org Sun Aug 26 20:09:47 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8335B1080E14 for ; Sun, 26 Aug 2018 20:09:47 +0000 (UTC) (envelope-from vixie@fsi.io) Received: from mail.fsi.io (mail.fsi.io [104.244.13.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2870B8F6D0 for ; Sun, 26 Aug 2018 20:09:47 +0000 (UTC) (envelope-from vixie@fsi.io) X-Virus-Scanned: amavisd-new at fsi.io Sender: vixie@fsi.io Received: from linux-9daj.localnet (dhcp-181.access.lah1.vix.su [24.104.150.181]) (Authenticated sender: vixie) by mail.fsi.io (Postfix) with ESMTPSA id 5CEE9608BC; Sun, 26 Aug 2018 20:09:42 +0000 (UTC) From: Paul Vixie To: freebsd-virtualization@freebsd.org, Paul Webster Subject: Re: Query regarding tutorials (Please have a quick read of me!) Date: Sun, 26 Aug 2018 20:09:41 +0000 Message-ID: <2265777.s6TpKQsPUA@linux-9daj> Organization: Farsight Security, Inc. In-Reply-To: References: <3626542.UnW4ZVN1pJ@linux-9daj> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 20:09:47 -0000 On Sunday, August 26, 2018 7:56:04 PM UTC Paul Webster via freebsd- virtualization wrote: > indeed I did polish it a bit and upload: https://youtu.be/w0WuoKVWAgI ;) "Your browser does not currently recognize any of the video formats available. Click here to visit our FAQ about HTML5 video." -- Vixie From owner-freebsd-virtualization@freebsd.org Sun Aug 26 20:21:11 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9F4D10813A1 for ; Sun, 26 Aug 2018 20:21:11 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-ed1-x541.google.com (mail-ed1-x541.google.com [IPv6:2a00:1450:4864:20::541]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B2658FEEB for ; Sun, 26 Aug 2018 20:21:10 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-ed1-x541.google.com with SMTP id b10-v6so9065796eds.4 for ; Sun, 26 Aug 2018 13:21:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=8BgQazHvGSu76M5Ysled+bydyUSpbt1DQGYng+R0C44=; b=UnK/02ivvqBgzYyuDAnIXvwIc7Z2XDtaatPYZvcTUZinjQ/vpKLNfxKLxbok+6Yx4I 0Y/8SSTLHqw49wJtB5kSkrEAPDY++Gka793naH2VZOAVALRYsUCJjSJoJIu7ykNdso9j 5SWmXkrAFwaZ+Ujyw6CHOA+jpZyF87kAMaRQdCgEu/wjRhwx9F1pgzeede5cyxBzrVEj 7dmcMBf7cHEqZnDT36EcFHGd8OTykKaeSfxbc80EA6cmtAB4fhxqUk2GMhzXMV2zcCFu bwCW/eg3sS0UbZuw2J8DtCYItEWppLgjVgX0qORCYo1Wf8M8yLVAXPU7bYHl+VsjW/58 jSSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=8BgQazHvGSu76M5Ysled+bydyUSpbt1DQGYng+R0C44=; b=KijhOABT9iljlackm/oftq/T0IZrVBQO8kLNI0ABvugXM14pAk69eM3e4kkSSV2az9 GMNV2JFWtiMkghhySrH1StRu8lI4nFi7T+TRVmMeE6EL53kGZZHKM40CLhqoV15V+vI7 GmwtEWhgjL/TjblRyu1c99IWVRo4dCkXBu4uCds0CdEnzLelgsi2vPhoOmD+wz1DXzNv nx0tNcdCKHVIek9qWd7/QRmk2WNzk/4P1VH57Jtbu/SO7EteJ88FdtOOtbfrsKXwd0L+ VFSfAKkCBwyTicFOFXQR9CdTdlyrORQY18kPMsBdtn4Le3pcGNtH6H0EETSZq1lNkYkP oheg== X-Gm-Message-State: APzg51BfFsvmuhtE+cpe5+GPLJbClsjpiTRhfqlKYH75k5vOUqbcp2sQ ZDi0LCWy50KVB/HWntMc3wCu2r5/hy4D0Q== X-Google-Smtp-Source: ANB0VdamzqYcEIu8FOfcsJF+XeveVmfQi/QAvDJ3igx6jzIozOzBcmDNRw0Ws8jxhtA08VLolRW/DA== X-Received: by 2002:a50:b0c1:: with SMTP id j59-v6mr13749928edd.267.1535314869878; Sun, 26 Aug 2018 13:21:09 -0700 (PDT) Received: from mutt-hbsd (tollana.enn.lu. [85.248.227.164]) by smtp.gmail.com with ESMTPSA id e30-v6sm5932978ede.91.2018.08.26.13.21.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 26 Aug 2018 13:21:09 -0700 (PDT) Date: Sun, 26 Aug 2018 16:20:14 -0400 From: Shawn Webb To: Paul Vixie Cc: freebsd-virtualization@freebsd.org, Paul Webster Subject: Re: Query regarding tutorials (Please have a quick read of me!) Message-ID: <20180826202014.pw2guxvwss4bvmqk@mutt-hbsd> References: <6699362.kGnQbBhLUH@linux-9daj> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="wsxh64wuyjhbvzcc" Content-Disposition: inline In-Reply-To: <6699362.kGnQbBhLUH@linux-9daj> X-Operating-System: FreeBSD mutt-hbsd 12.0-ALPHA2 FreeBSD 12.0-ALPHA2 X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0x6A84658F52456EEE User-Agent: NeoMutt/20180622 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 20:21:11 -0000 --wsxh64wuyjhbvzcc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Aug 25, 2018 at 05:45:52PM +0000, Paul Vixie wrote: > On Saturday, August 25, 2018 8:55:05 AM UTC Paul Webster via freebsd- > virtualization wrote: > > ... I personally have: > >=20 > > Windows Server 2012 > > CentOS 6 > > Debian > > FreeBSD > > OpenBSD > > and a few other more strange linuxs I believe (got a zpool full of the > > zvols (not named very well ;))) ... >=20 > i have no exotic linuxes, but two fairly modern debians. i remain mystifi= ed by=20 > the behaviour of grub-bhyve, which usually fails (no meaningful error mes= sage)=20 > and seems like won't run from /etc/rc.local (must be run from a shell.) s= o,=20 > one of my bhyve servers can't restart automatically, and i don't know why= =2E and=20 > i recently saw that the grub-bhyve package now lacks a maintainer, and ma= y be=20 > removed. I've found that the big distros (CentOS, Debian, Ubuntu) work very well with bhyve's UEFI support. It has been years since I used either bhyveload or grub-bhyve. CentOS does need a little massaging, renaming GRUBx64.efi to BOOTx64.efi or something like that. I can never remember off-hand. But other than renaming a file, Linux works great. :) Thanks, --=20 Shawn Webb Cofounder and Security Engineer HardenedBSD Tor-ified Signal: +1 443-546-8752 Tor+XMPP+OTR: lattera@is.a.hacker.sx GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE --wsxh64wuyjhbvzcc Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEKrq2ve9q9Ia+iT2eaoRlj1JFbu4FAluDC3kACgkQaoRlj1JF bu72zg/+Ihw2k6TqogHwTQijXrxrUrE1DKZ5JqtaFFRXujzQinP8DbEE1LY2iajE xQ3FJVQK8bR0ID7klOgxqWhyWT/Sbp1MoZoZLWZ0+imodPsfBtiWxXL5SoVa+2na ToYJ75gM3oKOlehvbKoy1+36BmnChOvKWve7BWpUXLWyAIF2oYwGW4+Fa6ZE3qL/ k4o0ZwTo8gwA5DP8GA6INzn7oxtX0tnyQYmntQvpzLZ3Rvrt2MRBuItcFL+3/i52 qdCsZtUnDE8IX88iivTM4qMZUK55PgFwbVNRzrbUQBsOCedTnHkvPNMG492JMKxU EwytnNLZ+2efGRTUaR6UiJK3VnyRYS/Us2ePGzrIlrqlS/EdZPQmJ868rwdtZGju 0ZwKJ727bnLv+Zxw06RY23higPcvFRKIE+N0x1gtkG2LMEGEZHLejSE79oGacK7O tW6MDVgnu2gHUO0q5f8ghhZrMKgnVpBr/iR2lbXXgl3wntqtjiecmzbox8m/e6zn ae2aoMkpV7UliXPfXhJgvWcmzgjBR8WUmrtuM3t4bsuDTIoxe7KCXsfwieaHelQf ZHwb0fBLGoMO3Zb2mhBj7B/v1rxEkEWBkQUeFJYZ85lpCeofrml3E0Mb6G3wbSLG QvBZSyimqfNiBeHIQI7tD2WHAE0351FK0w/pDh6R1eVCYc2qys4= =gfia -----END PGP SIGNATURE----- --wsxh64wuyjhbvzcc-- From owner-freebsd-virtualization@freebsd.org Sun Aug 26 20:27:23 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A83DF10815EE for ; Sun, 26 Aug 2018 20:27:23 +0000 (UTC) (envelope-from vixie@fsi.io) Received: from mail.fsi.io (mail.fsi.io [104.244.13.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 43B7D7008B for ; Sun, 26 Aug 2018 20:27:23 +0000 (UTC) (envelope-from vixie@fsi.io) X-Virus-Scanned: amavisd-new at fsi.io Sender: vixie@fsi.io Received: from linux-9daj.localnet (dhcp-181.access.lah1.vix.su [24.104.150.181]) (Authenticated sender: vixie) by mail.fsi.io (Postfix) with ESMTPSA id 8F1EF604F2; Sun, 26 Aug 2018 20:27:19 +0000 (UTC) From: Paul Vixie To: Shawn Webb Cc: freebsd-virtualization@freebsd.org, Paul Webster Subject: Re: Query regarding tutorials (Please have a quick read of me!) Date: Sun, 26 Aug 2018 20:27:18 +0000 Message-ID: <4392659.UZsOocES2z@linux-9daj> Organization: Farsight Security, Inc. In-Reply-To: <20180826202014.pw2guxvwss4bvmqk@mutt-hbsd> References: <6699362.kGnQbBhLUH@linux-9daj> <20180826202014.pw2guxvwss4bvmqk@mutt-hbsd> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 20:27:23 -0000 On Sunday, August 26, 2018 8:20:14 PM UTC Shawn Webb wrote: > ... > > I've found that the big distros (CentOS, Debian, Ubuntu) work very > well with bhyve's UEFI support. It has been years since I used either > bhyveload or grub-bhyve. i am trying alan jude's recommendation for UEFI bios now. > CentOS does need a little massaging, renaming GRUBx64.efi to > BOOTx64.efi or something like that. I can never remember off-hand. But > other than renaming a file, Linux works great. :) wait, what? where is that file? -- Vixie From owner-freebsd-virtualization@freebsd.org Sun Aug 26 23:06:30 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81A881085DFE for ; Sun, 26 Aug 2018 23:06:30 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: from mail-oi0-x22d.google.com (mail-oi0-x22d.google.com [IPv6:2607:f8b0:4003:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E73187543F for ; Sun, 26 Aug 2018 23:06:29 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: by mail-oi0-x22d.google.com with SMTP id 13-v6so24216811ois.1 for ; Sun, 26 Aug 2018 16:06:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=p+zpbDqjocNk0J1P7dKTAB66etUmsOzFGAeVMAubve0=; b=OQwB6u13N0cr0UiQrE7frYJP5fxrScQ9u54K3a3bnG3E/twh/T+hZpbgnCN5Ls64Tu 3H+9yW5JcSfMM9wCjgih5FxFib/tUfqsilrxb/FN9lPh7p+/skk1ywmhi30rFheOO/Ls FyudRR/PCw3i60/NduHkaJw9xS2av9HhOCpUw6lGz2C/brFtqyTBDuxotBeVC2Fog63p f5bTBjLhEDUwj9fXxdc7Mhsqbgr+1WyOlcg8qKzQ1Z9xQvlozNs1Y+LmCAc4VbkGfuF3 SdZCDqxMkqL3p8q1+HyxD9ISU+ydbPsl9AZsID3aQT97xDH3Ev143eDRSXWyUF5OPEpS zu1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding:content-language; bh=p+zpbDqjocNk0J1P7dKTAB66etUmsOzFGAeVMAubve0=; b=NXfljuWmBP/FSw3D8xXWecDkvv+z9pslFt7EhwGf/tzCZt0REm3/icdNhpe5yCMlM+ IUZ87Zm9JFbJfgW37GfgccxSYw/gMVdCApF7FTGquBxpbqGXXTUdBflpHxZiTt9tapDh uM3mKyQGrZYl5+wDYjUy1XM184dsCIQ3Y5QAnx+jEj9weqq/zvoQqaiGB11bjB1/s8qA 1s1B5wsbAgYJ2RV6qqC91fnIhMMgTZ2rgKoGSYu7+s5M95XmufxadkMVSDjdtRQ76Fvl X9RIg02xsuWNDDA79uq+aQaDMTQ9I6gT/Jkeap2P/TRJzwHK46Bc64dC6wQDnuYU2qE3 mHjg== X-Gm-Message-State: APzg51Ah46HFBFlZfRJwUDbuo9QtEX27zKIr3AZsXsxm4S/1J7BqJhMq Se6db5Pkzgs+VdqrSWcfYqrLYi7fWyg= X-Google-Smtp-Source: ANB0VdbgiyNSNXT8DEjVLjdhDmp8dmwav7uywtLYvSHABDhZKPo7xuH/VAQBa/OvYrEe9cqjBvgD7w== X-Received: by 2002:aca:3fc2:: with SMTP id m185-v6mr10533467oia.240.1535324788921; Sun, 26 Aug 2018 16:06:28 -0700 (PDT) Received: from ?IPv6:2601:2c2:800:300::ff24? ([2601:2c2:800:300::ff24]) by smtp.gmail.com with ESMTPSA id o125-v6sm21956071oig.44.2018.08.26.16.06.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 26 Aug 2018 16:06:28 -0700 (PDT) To: freebsd-virtualization From: Rajil Saraswat Subject: Bhyve guest boot error, Buffer I/O error on dev vda1, logical block 0, lost async page write Message-ID: Date: Sun, 26 Aug 2018 18:06:24 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2018 23:06:30 -0000 Hello, My bhyve guests are failing to boot. I upgraded from 11.1 to 11.2 recently, not sure if that is the cause. Here is an Ubuntu log, [    4.824240] md: raid6 personality registered for level 6 [    4.826021] md: raid5 personality registered for level 5 [    4.827780] md: raid4 personality registered for level 4 [    4.843091] md: raid10 personality registered for level 10 done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... [    4.913144] Btrfs loaded Scanning for Btrfs filesystems done. Begin: Will now check root file system ... fsck from util-linux 2.27.1 [/sbin/fsck.ext4 (1) -- /dev/vda1] fsck.ext4 -a -C0 /dev/vda1 /dev/vda1: recovering journal [    5.208906] blk_update_request: I/O error, dev vda, sector 2048 [    5.210997] Buffer I/O error on dev vda1, logical block 0, lost async page write [    5.213627] Buffer I/O error on dev vda1, logical block 1, lost async page write [    5.216231] blk_update_request: I/O error, dev vda, sector 4196688 [    5.218408] Buffer I/O error on dev vda1, logical block 524330, lost async page write [    5.221199] Buffer I/O error on dev vda1, logical block 524331, lost async page write [    5.223946] blk_update_request: I/O error, dev vda, sector 4196848 [    5.226143] Buffer I/O error on dev vda1, logical block 524350, lost async page write [    5.228913] Buffer I/O error on dev vda1, logical block 524351, lost async page write [    5.231681] blk_update_request: I/O error, dev vda, sector 4196480 [    5.233862] Buffer I/O error on dev vda1, logical block 524304, lost async page write [    5.236631] blk_update_request: I/O error, dev vda, sector 4196880 [    5.238821] Buffer I/O error on dev vda1, logical block 524354, lost async page write [    5.241571] blk_update_request: I/O error, dev vda, sector 4196360 [    5.243732] Buffer I/O error on dev vda1, logical block 524289, lost async page write [    5.246475] Buffer I/O error on dev vda1, logical block 524290, lost async page write [    5.249204] blk_update_request: I/O error, dev vda, sector 4196952 [    5.251385] blk_update_request: I/O error, dev vda, sector 4196616 [    5.253577] blk_update_request: I/O error, dev vda, sector 4196984 [    5.255748] blk_update_request: I/O error, dev vda, sector 4197000 [    5.263983] VFS: Dirty inode writeback failed for block device vda1 (err=-5). /dev/vda1: Superblock needs_recovery flag is clear, but journal has data. [    5.269118] VFS: Dirty inode writeback failed for block device vda1 (err=-5). /dev/vda1: Run journal anyway /dev/vda1: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.         (i.e., without -a or -p options) fsck exited with status code 4 done. Failure: File system check of the root filesystem failed The root filesystem on /dev/vda1 requires a manual fsck Any idea what could be the issue? From owner-freebsd-virtualization@freebsd.org Mon Aug 27 00:23:48 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E7D110876D7 for ; Mon, 27 Aug 2018 00:23:48 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2BAB576FF3 for ; Mon, 27 Aug 2018 00:23:48 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: by mail-oi0-x231.google.com with SMTP id r69-v6so23765618oie.3 for ; Sun, 26 Aug 2018 17:23:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:references:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding:content-language; bh=uhv53ped757GP4nnEutv+ctmq17/QfpasHr4bpHVbqM=; b=IzdU94bFyEdlSOaz78TH3XIwDoslZedWc14KfeR8kulWLq5tBl0N8BW1zIkYnFEyur mnaP+tCAQXsOppf6d1Vn3VA4ikjYWR6zifdUz65qDroe1q84F4jwiBFXik/89dQQpJta TWTHww6fNRiKGO343xFeVIb6SK+LRfibp7tWiZP2/4Y8gcm5Wikjhm1tkcNWGn15hAjC aNQoepvTz2KXO5c8DCg7WmwVJE6S+k1wVcD24gZh7P6FHR60AkIV4DGMtoLKjC5nNBoZ zIBdXvbMA1HSylel8jzG4gyJbwRNPjAtBtSwxvaKyObQO5hx/3Ws9CW8npc1nFsoNVGN bseg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:references:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=uhv53ped757GP4nnEutv+ctmq17/QfpasHr4bpHVbqM=; b=DaDVug8I3JEoO4vQ4MHXl/aD7zZaOJNyp0WDaAQ7mDl8t4Z9AnPkpeOHMkWmEIzvhX +QjFbF0DBm8aEZS08Tu4fQzkv5nKj6hokpki+5yGwLSP4otG7UkThJIun5wfAIGUBMcB zXDQmLbRKJX/wUfhgUtWtImDAylae3jKytZQvGk2Rrk6/iDbXK9fJT96Foosglm8ruj/ 2Nft7Vtz1cxt+jXK/xMdNCnjU5MlwVKNF6ssK0qunRJasV0Fqt/8p7OOH1qmhcxYHG7+ vKdlLTcDe1/nryFNzf+8HzobebQODL4pak8COcDYEF06GwCZkZm47/bwCtntnVSmxq/e pzHg== X-Gm-Message-State: APzg51ANfET7tbmeIHBmCWpHh4h07auqKOeyqlzt9EJOmyGHkeGv8EAT BFnpttxY/MfrFoXFeKfB3IUrQAk3IVI= X-Google-Smtp-Source: ANB0VdahejYKtAKcIqBzmovLlAPZddtKDaA1ydaw7/QMm7geE7V+vTE6Fe8fUr6MMMXPiTjjdcEtGw== X-Received: by 2002:aca:cc04:: with SMTP id c4-v6mr11074017oig.314.1535329427101; Sun, 26 Aug 2018 17:23:47 -0700 (PDT) Received: from ?IPv6:2601:2c2:800:300::ff24? ([2601:2c2:800:300::ff24]) by smtp.gmail.com with ESMTPSA id t141-v6sm29554680oif.18.2018.08.26.17.23.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 26 Aug 2018 17:23:46 -0700 (PDT) Subject: Re: Bhyve guest boot error, Buffer I/O error on dev vda1, logical block 0, lost async page write From: Rajil Saraswat To: freebsd-virtualization References: Message-ID: <114c2000-17ba-3cf6-1d4a-1354f42d8875@gmail.com> Date: Sun, 26 Aug 2018 19:23:42 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 00:23:48 -0000 On 08/26/2018 06:06 PM, Rajil Saraswat wrote: > Hello, > > My bhyve guests are failing to boot. I upgraded from 11.1 to 11.2 > recently, not sure if that is the cause. > Any idea what could be the issue? > Figured out this one. Seems my pool had reached a capacity of 69%. After deleting unnecessary zvols, and reducing the pool to 52% the guest booted up fine. From owner-freebsd-virtualization@freebsd.org Mon Aug 27 05:08:02 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 760F9108C731; Mon, 27 Aug 2018 05:08:02 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-qt0-x22c.google.com (mail-qt0-x22c.google.com [IPv6:2607:f8b0:400d:c0d::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D9A77E0E0; Mon, 27 Aug 2018 05:08:01 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: by mail-qt0-x22c.google.com with SMTP id o15-v6so16641276qtk.6; Sun, 26 Aug 2018 22:08:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3P3z6fO2RrnrhjXgZp+XpBBEOudswwkpJn9jxe4jW6E=; b=TOO6n5xQLNfgKGH0Ct+gaRC3hEZgnsPhXu96sx/97WA9939HNlCdPddXv5vftvdKBP HS1hzC/tW4aGR9RgnPQwyePcrplFCzX1sJclao2LGRb2rqDuBOws+XPbFsvmYeyk3j6C UwWmTbDp6mHUmahru8Cjm3awRx1f2DsvfKEQ8EZ5j0ySZ8WHyHPNivZXHZaqEqfPBzul tTJZ4ZWUGCNfgVTHVizLOrj/3GktVnT6K6VeWypsX0ihXK/nZ2Qr4Jg0Ck+35JAEcdQ/ AYesFdKDf6kdynKETuJlsbWV/jfzLSFS0IliCEH7XttKAgtPjt3tcQ9Me0gx9Rn8Q+sK WM4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3P3z6fO2RrnrhjXgZp+XpBBEOudswwkpJn9jxe4jW6E=; b=hBIbeXdqh/xps05k2zBMYN5PSs+B+HdUdQGARAPiaTlCrviK6subVpRn8Wzk1m30Nj QsHZf5Aiqn/0Q6B67eJTDJzxq2paLwqHwoSuGzK/S7qN8IYzjB62qzjTsj0AemFMq6Hi +yxnGIkIyRDXem2LZCRJkERfhp4Ob6ZSzxvIAAZd8yp+oqEQS/tTDeK2SIQJUQA02+7k 136SkJQ13KXhK5EIeP679XysuKwg+1bJrSAXjc8W1Vy4M0rVbv9NE79tTIMlICSm8GnW qEJJGV8s8A0EhsGRpSNKoORVGxhbJF8Y4NK8z+VrvuWfHUoPLvfcxPQsVgBcF3jkoUpu Og8Q== X-Gm-Message-State: APzg51AtLH/OvZnjjc1q63B8NxDTT1vVk5RRi6cppTPcEOtHxXT1k9Gu InvIL9mOCK+yoEPLVNUSJ6BnKzYS3ulAnpycqr0= X-Google-Smtp-Source: ANB0VdYKgSHIHuzKzCagTkdi6Cwa3j3T5JqZq0l3LM7m9sah6RZ7oBwclLp0n2/75yOlwGH6juCO9ui2YngvfBgCjFI= X-Received: by 2002:a0c:869b:: with SMTP id 27-v6mr11972513qvf.92.1535346481417; Sun, 26 Aug 2018 22:08:01 -0700 (PDT) MIME-Version: 1.0 References: <114c2000-17ba-3cf6-1d4a-1354f42d8875@gmail.com> In-Reply-To: <114c2000-17ba-3cf6-1d4a-1354f42d8875@gmail.com> From: Sami Halabi Date: Mon, 27 Aug 2018 08:07:45 +0300 Message-ID: Subject: Re: Bhyve guest boot error, Buffer I/O error on dev vda1, logical block 0, lost async page write To: Rajil Saraswat , freebsd-fs@freebsd.org Cc: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 05:08:02 -0000 Hi, IIRC pool shouldn't be above 90% used otherwise bad things happen.. but at 69%???? Can someone explain thid please? Sami =D7=91=D7=AA=D7=90=D7=A8=D7=99=D7=9A =D7=99=D7=95=D7=9D =D7=91=D7=B3, 27 = =D7=91=D7=90=D7=95=D7=92=D7=B3 2018, 03:24, =D7=9E=D7=90=D7=AA Rajil Sarasw= at =E2=80=8F: > On 08/26/2018 06:06 PM, Rajil Saraswat wrote: > > Hello, > > > > My bhyve guests are failing to boot. I upgraded from 11.1 to 11.2 > > recently, not sure if that is the cause. > > Any idea what could be the issue? > > > Figured out this one. Seems my pool had reached a capacity of 69%. After > deleting unnecessary zvols, and reducing the pool to 52% the guest > booted up fine. > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@freebsd.org Mon Aug 27 14:15:46 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 981BE108B394 for ; Mon, 27 Aug 2018 14:15:46 +0000 (UTC) (envelope-from fabian.freyer@physik.tu-berlin.de) Received: from mail2.physik.tu-berlin.de (mail2.physik-pool.tu-berlin.de [130.149.50.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3857F7232A; Mon, 27 Aug 2018 14:15:45 +0000 (UTC) (envelope-from fabian.freyer@physik.tu-berlin.de) Received: from [130.149.50.202] (yalow02.physik-pool.tu-berlin.de [130.149.50.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail2.physik.tu-berlin.de (Postfix) with ESMTPSA id BB5AB61F92; Mon, 27 Aug 2018 14:15:37 +0000 (UTC) To: freebsd-virtualization@freebsd.org From: Fabian Freyer Subject: on bhyve statistics Message-ID: Date: Mon, 27 Aug 2018 16:14:46 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: de-DE Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 14:15:46 -0000 Hi list, I'm currently looking at getting the libvirt prometheus exporter[1] to work with libvirt+bhyve. In its current state this doesn't work because at least one of the API calls exposed by libvirt isn't implemented by the libvirt bhyve driver - so I started looking at implementing it. The first API call in question is virDomainBlockStats[2], which returns statistics (number of read and written bytes and ops, respectively). Currently, bhyve does not expose any of these statistics. All the stats available through bhyvectl --get-stats seem to be coming from the VMM, not from the userspace emulation. OTOH, I did see that there are *some* stats being collected in bhyverun.c (see struct bhyvestats {...} stats;). I can't see how these are exposed though - a grep of /usr/src turned up no other uses. Which brings me to the following questions: - are the stats in struct bhyvestats {...} stats exposed or used in any non-obvious way? - architecturally, what would be the best ways to get stats out of the user-space emulations? Off of the top of my head, I could think of the following possibilities: - prometheus exporter - having some socket or pipe to request them - DTrace probes I wouldn't mind implementing any of the above, and so would like to know which of these (or other options) would be the most acceptable, and would appreciate some guidance. CC'ing novel@ for the libvirt side, and grehan@ for the architectural bhyve questions. Fabian [1] https://github.com/kumina/libvirt_exporter [2] https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockStats From owner-freebsd-virtualization@freebsd.org Mon Aug 27 14:34:35 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1CF0108C1D2 for ; Mon, 27 Aug 2018 14:34:35 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B991736D2 for ; Mon, 27 Aug 2018 14:34:35 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lf1-x134.google.com with SMTP id z11-v6so12118295lff.9 for ; Mon, 27 Aug 2018 07:34:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:cc; bh=P7S4scJKt66rTV3PygK2EgT6tzpg1vvcH2TNuuvZ2Sk=; b=XI9pZoPMYoi8HixUO66RadODSpLLfxzsnF84DmbgHHP4vPBqNY8RP7I4ykcGk8xkxE eahZFULmcKv0YtQK7H7R71nz2nMveKJRwDPLpJhSgiJtaqf6JIvm7WgCFz2kSYmyjyu4 QrNxMiJBoUKy729c4e8TtcazAg7RkytnsVUX4yOog9VEj2H3vdHBnGSrY7lsgtkEfoxd ORqmjxhHHyIRC3ZlScVeHWv6Ig0waE4t3W8rhO496FngJ3UXCAwTbFf4+3djTCGyg+Cy nxxufb/NWpmytO6pSkG1sBi3KxRUL6wsFLVblq+8RTKBYN9WCLenZ5CZiwHuolejRH8w J+SA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=P7S4scJKt66rTV3PygK2EgT6tzpg1vvcH2TNuuvZ2Sk=; b=fowtNWf50XDqeVx8iOneWIVfoiCPAyactIH+YodHCKXphMZ+UXsZEBKB7YhuBxXM0F IC0/woIhPN1g7xaPnuYtBz8AXigY6HeCCJwogGz5iMQlAYR/H08ltSbGpZ3EQ1Ou1FEd NRjdntyycQPSsoz04QLpsKriPwcML1qLnmTTDT5Xr+VFqvUmFJdf/ic/JlMuPAljZ0wu j574eMZ1bU5/eC/L5HOPvsWebZC7BBhqaQQeXzN/X0FdJykHMeNJRe3ekqlNmwAH7IPw hecw9jUFNSmb7/iQcKL+WEVIzrnEzR1jYeMo9l77gm0uplq1zoN+FDooKStOAYL0kv4d cUEg== X-Gm-Message-State: APzg51DBWAZMNxCknlKszFRCDioZ8mpziQ95e9waS5R+Qdc9AD9upl5I CkkxA6ilThLWSe6Jb4PrYsomDRtRgZ+/ni2GxFg= X-Google-Smtp-Source: ANB0VdbrQ9uKFNLI01c7/4bUwMwwnqrip9HX+dHBib2Bk9bk4bFaYALCza71fkOWIpyRBK8mjtTKU2cQ8kgOMU1yUf0= X-Received: by 2002:a19:6308:: with SMTP id x8-v6mr8317614lfb.140.1535380473813; Mon, 27 Aug 2018 07:34:33 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a19:1f4c:0:0:0:0:0 with HTTP; Mon, 27 Aug 2018 07:34:32 -0700 (PDT) Reply-To: araujo@freebsd.org In-Reply-To: References: From: Marcelo Araujo Date: Mon, 27 Aug 2018 22:34:32 +0800 Message-ID: Subject: Re: on bhyve statistics To: Fabian Freyer Cc: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 14:34:35 -0000 Hi Fabian, 2018-08-27 22:14 GMT+08:00 Fabian Freyer : > Hi list, > > I'm currently looking at getting the libvirt prometheus exporter[1] to > work with libvirt+bhyve. In its current state this doesn't work because > at least one of the API calls exposed by libvirt isn't implemented by > the libvirt bhyve driver - so I started looking at implementing it. > > The first API call in question is virDomainBlockStats[2], which returns > statistics (number of read and written bytes and ops, respectively). > > Currently, bhyve does not expose any of these statistics. All the stats > available through bhyvectl --get-stats seem to be coming from the VMM, > not from the userspace emulation. OTOH, I did see that there are *some* > stats being collected in bhyverun.c (see struct bhyvestats {...} > stats;). I can't see how these are exposed though - a grep of /usr/src > turned up no other uses. Which brings me to the following questions: > > - are the stats in struct bhyvestats {...} stats exposed or used in any > non-obvious way? > They are most used inside bhyverun by the name "stats" and the purpose is not really to collect statistics about the guest vm. > > - architecturally, what would be the best ways to get stats out of the > user-space emulations? Off of the top of my head, I could think of the > following possibilities: > - prometheus exporter > - having some socket or pipe to request them > - DTrace probes > I don't know what kind of stats do you need to collect, but based on the assumptions you listed above, maybe you can take a look at this project: https://github.com/freenas/bhyve-vm-goagent bhyve-vm-goagent can easily be extended to collect other information from guest. Although I'm not sure if it will be useful for your case, looks like you are looking for something less intrusive than bhyve-vm-goagent. > > I wouldn't mind implementing any of the above, and so would like to know > which of these (or other options) would be the most acceptable, and > would appreciate some guidance. > > CC'ing novel@ for the libvirt side, and grehan@ for the architectural > bhyve questions. > > Fabian > > [1] https://github.com/kumina/libvirt_exporter > [2] https://libvirt.org/html/libvirt-libvirt-domain.html# > virDomainBlockStats > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization- > unsubscribe@freebsd.org" > -- -- Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-virtualization@freebsd.org Mon Aug 27 14:56:23 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8936F108CDF4 for ; Mon, 27 Aug 2018 14:56:23 +0000 (UTC) (envelope-from fabian.freyer@physik.tu-berlin.de) Received: from mail2.physik.tu-berlin.de (mail2.physik-pool.tu-berlin.de [130.149.50.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2A8377442A; Mon, 27 Aug 2018 14:56:23 +0000 (UTC) (envelope-from fabian.freyer@physik.tu-berlin.de) Received: from [130.149.50.202] (yalow02.physik-pool.tu-berlin.de [130.149.50.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail2.physik.tu-berlin.de (Postfix) with ESMTPSA id 29A9461F92; Mon, 27 Aug 2018 14:56:22 +0000 (UTC) Subject: Re: on bhyve statistics To: Marcelo Araujo References: From: Fabian Freyer Cc: freebsd-virtualization@freebsd.org Message-ID: <26beeb83-1e62-af39-5532-680062ee7d7d@physik.tu-berlin.de> Date: Mon, 27 Aug 2018 16:56:21 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: de-DE Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 14:56:23 -0000 On 27.08.2018 16:34, Marcelo Araujo wrote: > Hi Fabian, > > 2018-08-27 22:14 GMT+08:00 Fabian Freyer > >: [...] > - are the stats in struct bhyvestats {...} stats exposed or used in any >   non-obvious way? > > > They are most used inside bhyverun by the name "stats" and the purpose > is not really to collect statistics about the guest vm. What are they used for? I only see them get increased, never actually checked: $ rg stats /usr/src/usr.sbin/bhyve /usr/src/usr.sbin/bhyve/bhyverun.c 117:struct bhyvestats { 126:} stats; 510: stats.vmexit_bogus++; 521: stats.vmexit_reqidle++; 530: stats.vmexit_hlt++; 544: stats.vmexit_pause++; 555: stats.vmexit_mtrap++; 566: stats.vmexit_inst_emul++; > - architecturally, what would be the best ways to get stats out of the >   user-space emulations? Off of the top of my head, I could think of the >   following possibilities: >   - prometheus exporter >   - having some socket or pipe to request them >   - DTrace probes > > > I don't know what kind of stats do you need to collect, but based on the > assumptions you listed above, maybe you can take a look at this project: > https://github.com/freenas/bhyve-vm-goagent > > bhyve-vm-goagent can easily be extended to collect other information > from guest. Although I'm not sure if it will be useful for your case, > looks like you are looking for something less intrusive than > bhyve-vm-goagent. Thanks for the pointer - but you're right, I'm looking for guest-agnostic monitoring on the host. Fabian From owner-freebsd-virtualization@freebsd.org Mon Aug 27 15:00:28 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 381C5108CF8A for ; Mon, 27 Aug 2018 15:00:28 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: from mail-qk0-x236.google.com (mail-qk0-x236.google.com [IPv6:2607:f8b0:400d:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CCDDC745DD for ; Mon, 27 Aug 2018 15:00:27 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: by mail-qk0-x236.google.com with SMTP id z125-v6so10647251qkb.12 for ; Mon, 27 Aug 2018 08:00:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=Bd6MYLRmuDIvTvwYi+VuKozFPjPnId6tsnxDrtO5xtM=; b=Mq+T45BgDn/AkXV4Uj5ZR+un3STeJ1WDOeQj3+yJf1NMGKw/RGxSG8ecm8Z2sEJaAb aUpTtWYzHRYE7NxWk+gT7jYDKqT2db/ApwZXRzKoj/l510ZB5f/iO/4s2ydCCtQoMFwo Yf5jgqiXstSmAnCooPbvjJqxML+Jtll+WryipWAnm8ZrSZkQJU0IrYC0puikieFrZl+/ dLRzyhFTEiUgBb50tvZZjn24Pg/9urem6RYVvEj1Bc6sEfKCpJjxiOGs8U/rJNBmwfwo Fv1Nlz05cNyyFZKkReKVQPeoPsiLAwmu2f94ASuxUnv+dj1QagIuziziiuGGtJlMBFzo 96LQ== X-Gm-Message-State: APzg51CIVm/t5O3tXjWRkmKoFQMzNjilYK4aA6uVI3Z3sRmFi2KYRHpR s3jvKGxa0POV0NKncp5l3Dgl7pOFwYFgBaC41wbUFmVhmHY= X-Google-Smtp-Source: ANB0VdawDcTShyP2S3C0jpWmJeFca/QOV99v8DbZvkLbtWHfijAZNAl4BElrVkpRR/lOTSMPLubrTan1TBbSONJcC9U= X-Received: by 2002:a37:4647:: with SMTP id t68-v6mr13892254qka.260.1535382026715; Mon, 27 Aug 2018 08:00:26 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Paul Webster Date: Mon, 27 Aug 2018 16:00:15 +0100 Message-ID: Subject: Re: Query regarding tutorials (Please have a quick read of me!) To: "freebsd-virtua." Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 15:00:28 -0000 Heya all part 2 'windows server 2016' is ready: https://www.youtube.com/watch?v=tHIWdoNXGec On Sat, 25 Aug 2018 at 09:55, Paul Webster wrote: > Hey all, > > A while ago I created a youtube channel to start doing video example > versions of the FreeBSD documentation, I have decided that this weekend I > would take a swing at showing howto setup a Bhyve VM, I personally have: > > Windows Server 2012 > CentOS 6 > Debian > FreeBSD > OpenBSD > and a few other more strange linuxs I believe (got a zpool full of the > zvols (not named very well ;))) anyhow IRC tells me that a lot of users > have trouble getting windows server working. I know obviously that 2012 > works but I am willing to take a swing at 2016 if people would want that. > > Or would the community benefit more of an example of 2012 (which is much > lighter)? > > Or perhaps something entirely different, any replies would be more than > welcome :) > From owner-freebsd-virtualization@freebsd.org Mon Aug 27 15:20:01 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8634F108D9DB for ; Mon, 27 Aug 2018 15:20:01 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CA699753A7 for ; Mon, 27 Aug 2018 15:20:00 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7RFJupE011001; Mon, 27 Aug 2018 08:19:56 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7RFJt5J011000; Mon, 27 Aug 2018 08:19:55 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808271519.w7RFJt5J011000@pdx.rh.CN85.dnsmgr.net> Subject: Re: on bhyve statistics In-Reply-To: To: Fabian Freyer Date: Mon, 27 Aug 2018 08:19:55 -0700 (PDT) CC: freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 15:20:01 -0000 > Hi list, > > I'm currently looking at getting the libvirt prometheus exporter[1] to > work with libvirt+bhyve. In its current state this doesn't work because > at least one of the API calls exposed by libvirt isn't implemented by > the libvirt bhyve driver - so I started looking at implementing it. > > The first API call in question is virDomainBlockStats[2], which returns > statistics (number of read and written bytes and ops, respectively). > > Currently, bhyve does not expose any of these statistics. All the stats > available through bhyvectl --get-stats seem to be coming from the VMM, > not from the userspace emulation. That is correct, byhvectl is a diagnostics tool for getting information from the kernel/vmm module. > OTOH, I did see that there are *some* > stats being collected in bhyverun.c (see struct bhyvestats {...} > stats;). I can't see how these are exposed though - a grep of /usr/src > turned up no other uses. Which brings me to the following questions: > > - are the stats in struct bhyvestats {...} stats exposed or used in any > non-obvious way? Not that I am aware of. > - architecturally, what would be the best ways to get stats out of the > user-space emulations? Off of the top of my head, I could think of the > following possibilities: > - prometheus exporter > - having some socket or pipe to request them > - DTrace probes > > I wouldn't mind implementing any of the above, and so would like to know > which of these (or other options) would be the most acceptable, and > would appreciate some guidance. I differ to others on what may be the best way to do this. > CC'ing novel@ for the libvirt side, and grehan@ for the architectural > bhyve questions. You should replace @grehan with @jhb,@tychon as Peter has moved on, and John and Tycho are now the bhyve maintainers. I was going to add them, and remove Peter, but I see no cc: anyway, so I am sure that they are on the virtualization list though. > Fabian > > [1] https://github.com/kumina/libvirt_exporter > [2] https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockStats > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Mon Aug 27 15:20:17 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB7BC108D9F5 for ; Mon, 27 Aug 2018 15:20:17 +0000 (UTC) (envelope-from tommi.pernila@gmail.com) Received: from mail-qk0-f175.google.com (mail-qk0-f175.google.com [209.85.220.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6587B753C0 for ; Mon, 27 Aug 2018 15:20:17 +0000 (UTC) (envelope-from tommi.pernila@gmail.com) Received: by mail-qk0-f175.google.com with SMTP id g13-v6so7360846qki.9 for ; Mon, 27 Aug 2018 08:20:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HIi7Tp2s74+mtF9nDmmfEHKOqFMguGxHNYUOmLCCj8w=; b=FiJ49fdNIWk3BBhYjOJMnZzLobJ0aixlrkjJab0PvSPaprUcZF4yfvrKZlKdOhLbuM qEsvOnUUKKBw4X930nbiyQt5WeZwTUgll8/5N26UJHz/9n70fEICwxKtQVd7vnsf1JP8 N/0iUhgBn7AYUvm3XwqHI6coWQbuf6ER+q5Ys102AKpqu62ptFp/yaOR65CybME651uH IehqlucHyh2tA8KefAv6JZT3Rtko8ovB+H8w8Im5+lPv928uilCylLoUdPMdXsTiWHjT DxsZA0mB+hxsxJvnyiGc5m0bjuqeAybgoDUqNuhYEGcZGxxzdwo+4k9dpEd8PUH7ukOA cEbA== X-Gm-Message-State: APzg51DMrprnF3C99EACJKztTGUCwoSHzSuWr0QGAY0VKzswBlXldZgM Nctxr9pah/KWqeY0165YHNJEYcv1bcscKMJjaLw= X-Google-Smtp-Source: ANB0VdYztLEtDlXU9GJ9hluyEU6tfcAzbBVySosjAPpG/R2zhs2LUvIDf60uIzqc31D64/OK2x53FnzI4W/Wo+3t6Ew= X-Received: by 2002:a37:d708:: with SMTP id m8-v6mr13246702qki.132.1535382844933; Mon, 27 Aug 2018 08:14:04 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Tommi Pernila Date: Mon, 27 Aug 2018 18:13:53 +0300 Message-ID: Subject: Re: Query regarding tutorials (Please have a quick read of me!) To: Paul Webster Cc: "freebsd-virtua." Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 15:20:18 -0000 Hi Paul, thanks for the videos, they are looking great. some feedback: Do keep the small mistakes like typoes or similar in the video. It's always interesting to see how people correct the errors. In the first video the left side of the terminal was slightly cut out. Not too much though. In the second video your webcam footage is partly blocking the view. Maybe split the right side vertically with tmux? Looking forward to more of your videos. Br, Tommi On Mon, 27 Aug 2018 at 18.01, Paul Webster via freebsd-virtualization < freebsd-virtualization@freebsd.org> wrote: > Heya all part 2 'windows server 2016' is ready: > > https://www.youtube.com/watch?v=tHIWdoNXGec > > On Sat, 25 Aug 2018 at 09:55, Paul Webster > wrote: > > > Hey all, > > > > A while ago I created a youtube channel to start doing video example > > versions of the FreeBSD documentation, I have decided that this weekend I > > would take a swing at showing howto setup a Bhyve VM, I personally have: > > > > Windows Server 2012 > > CentOS 6 > > Debian > > FreeBSD > > OpenBSD > > and a few other more strange linuxs I believe (got a zpool full of the > > zvols (not named very well ;))) anyhow IRC tells me that a lot of users > > have trouble getting windows server working. I know obviously that 2012 > > works but I am willing to take a swing at 2016 if people would want that. > > > > Or would the community benefit more of an example of 2012 (which is much > > lighter)? > > > > Or perhaps something entirely different, any replies would be more than > > welcome :) > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@freebsd.org Mon Aug 27 15:35:26 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7530F108E05B for ; Mon, 27 Aug 2018 15:35:26 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: from mail-qt0-x234.google.com (mail-qt0-x234.google.com [IPv6:2607:f8b0:400d:c0d::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1861675E31 for ; Mon, 27 Aug 2018 15:35:26 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: by mail-qt0-x234.google.com with SMTP id d4-v6so18453423qtn.13 for ; Mon, 27 Aug 2018 08:35:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/IrVYKPTgrHtX6zsOR4PJqoswhZ5J+0ePEhyhrtbA5w=; b=Gf3jQb79zGFQd+SEX8IANHC9GaD91w6PMu7dZWI0xwh8FUu3PApz0ok3yeSEXJbwOd jo664nWU7HrlA1BDoLXIO/ihlqwQq/jt1T7nGGLqgkgYlGH5QE0NYnfcgo6jafYJEaHC ku+gy/a4it0572qUDw3ImCRVHxdjedRo2pC1zu9sDZXID6TJOVigYfiaQC3NfY/Uq2Ba QMgNIp2KGTExHIOMCSMEo2Bx+GRLxvrhNgpp1qQ2547UJ8IcIyVe6ID7W/ildI1JaEPq FCs4XqM4kaosEY+Dm1nTOz7JqxeNWwvylDG9ZiIUdvfHu4rKu88bPBX/vdvxbbsU6t+e 1ahg== X-Gm-Message-State: APzg51DybX6GzZc7LS7z/Rg0DyrX2VqfmqEOLXHzdV4ewwXDTwjm+zr1 qlbYsTV9NTbCvG8UpL3nRyjzTzkrWtr4Ntedy90= X-Google-Smtp-Source: ANB0VdYUYqhpsqnk1D6oTHoYop78MqfffvVnqFHgolTCVuLI5OgMT/5Pamb/56EN4fiioouIfKtwbUWMw8bxAOnAKJQ= X-Received: by 2002:a0c:da8e:: with SMTP id z14-v6mr13896758qvj.171.1535384125502; Mon, 27 Aug 2018 08:35:25 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Paul Webster Date: Mon, 27 Aug 2018 16:35:14 +0100 Message-ID: Subject: Re: Query regarding tutorials (Please have a quick read of me!) To: tommi.pernila@iki.fi Cc: "freebsd-virtua." Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 15:35:26 -0000 Thanks for the feedback! Indeed I quite like the errors too when I see other people, its nice when its not to 'rehearsed' feels more natural :) On Mon, 27 Aug 2018 at 16:14, Tommi Pernila wrote: > Hi Paul, > > thanks for the videos, they are looking great. > > some feedback: > Do keep the small mistakes like typoes or similar in the video. It's > always interesting to see how people correct the errors. > > In the first video the left side of the terminal was slightly cut out. Not > too much though. > > In the second video your webcam footage is partly blocking the view. Maybe > split the right side vertically with tmux? > > Looking forward to more of your videos. > > > Br, > > Tommi > > On Mon, 27 Aug 2018 at 18.01, Paul Webster via freebsd-virtualization < > freebsd-virtualization@freebsd.org> wrote: > >> Heya all part 2 'windows server 2016' is ready: >> >> https://www.youtube.com/watch?v=tHIWdoNXGec >> >> On Sat, 25 Aug 2018 at 09:55, Paul Webster > > >> wrote: >> >> > Hey all, >> > >> > A while ago I created a youtube channel to start doing video example >> > versions of the FreeBSD documentation, I have decided that this weekend >> I >> > would take a swing at showing howto setup a Bhyve VM, I personally have: >> > >> > Windows Server 2012 >> > CentOS 6 >> > Debian >> > FreeBSD >> > OpenBSD >> > and a few other more strange linuxs I believe (got a zpool full of the >> > zvols (not named very well ;))) anyhow IRC tells me that a lot of users >> > have trouble getting windows server working. I know obviously that 2012 >> > works but I am willing to take a swing at 2016 if people would want >> that. >> > >> > Or would the community benefit more of an example of 2012 (which is much >> > lighter)? >> > >> > Or perhaps something entirely different, any replies would be more than >> > welcome :) >> > >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to " >> freebsd-virtualization-unsubscribe@freebsd.org" >> > From owner-freebsd-virtualization@freebsd.org Mon Aug 27 16:50:28 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4CFE510901B9; Mon, 27 Aug 2018 16:50:28 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from vps-mail.nomadlogic.org (mail.nomadlogic.org [IPv6:2607:f2f8:a098::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C4E479744; Mon, 27 Aug 2018 16:50:27 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from [192.168.1.106] (cpe-76-175-75-27.socal.res.rr.com [76.175.75.27]) by vps-mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id dced2b94 TLS version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO; Mon, 27 Aug 2018 09:50:25 -0700 (PDT) Subject: Re: Bhyve guest boot error, Buffer I/O error on dev vda1, logical block 0, lost async page write To: Sami Halabi , Rajil Saraswat , freebsd-fs@freebsd.org Cc: FreeBSD virtualization References: <114c2000-17ba-3cf6-1d4a-1354f42d8875@gmail.com> From: Pete Wright Message-ID: Date: Mon, 27 Aug 2018 09:50:23 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 16:50:28 -0000 On 8/26/18 10:07 PM, Sami Halabi wrote: > Hi, > IIRC pool shouldn't be above 90% used otherwise bad things happen.. but at > 69%???? > Can someone explain thid please? it's possible the filesystem was heavily fragmented?  seems suspicious - did scrubbing your pool find any errors? -pete -- Pete Wright pete@nomadlogic.org @nomadlogicLA From owner-freebsd-virtualization@freebsd.org Mon Aug 27 21:33:40 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D2D11096C79 for ; Mon, 27 Aug 2018 21:33:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CF0E28439F for ; Mon, 27 Aug 2018 21:33:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 932C41096C78; Mon, 27 Aug 2018 21:33:39 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 820AE1096C77 for ; Mon, 27 Aug 2018 21:33:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 235ED8439D for ; Mon, 27 Aug 2018 21:33:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 5747D192A9 for ; Mon, 27 Aug 2018 21:33:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7RLXcUU094771 for ; Mon, 27 Aug 2018 21:33:38 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7RLXcjP094770 for virtualization@FreeBSD.org; Mon, 27 Aug 2018 21:33:38 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 229167] [Hyper-V] [Jun 19, 2018] Recently FreeBSD VM panics during boot-up, especially with Mellanox VF configured Date: Mon, 27 Aug 2018 21:33:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: decui@microsoft.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 21:33:40 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229167 --- Comment #32 from Dexuan Cui --- (In reply to Andriy Gapon from comment #31) Unluckily I still can't reproduce the issue recently. :-( I created the Marketplace 11.1 VM on Azure, and can't reproduce the issue: FreeBSD decui-bsd111 11.1-RELEASE-p6 FreeBSD 11.1-RELEASE-p6 #0 r313908+f9756700e2c(releng/11.1)-dirty: Thu Jan 11 23:23:54 CST 2018=20=20= =20=20 root@bsd:/usr/obj/usr/xhx/freebsd/sys/GENERIC amd64 Next, I upgraded it to the latest 11.1, and still no luck: FreeBSD decui-bsd111 11.1-RELEASE-p13 FreeBSD 11.1-RELEASE-p13 #0: Tue Aug = 14 19:34:21 UTC 2018=20=20=20=20 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 Finally, I upgraded it to the latest 11.2, and still no luck: FreeBSD decui-bsd111 11.2-RELEASE-p2 FreeBSD 11.2-RELEASE-p2 #0: Tue Aug 14 21:45:40 UTC 2018=20=20=20=20 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 Please let us know if you can still reproduce the issue, especially with the same latest 11.1 and 11.2 releases. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Aug 27 22:42:18 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 915CF109837E for ; Mon, 27 Aug 2018 22:42:18 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: from mail-oi0-x22c.google.com (mail-oi0-x22c.google.com [IPv6:2607:f8b0:4003:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2A5F986790 for ; Mon, 27 Aug 2018 22:42:18 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: by mail-oi0-x22c.google.com with SMTP id p84-v6so1140290oic.4 for ; Mon, 27 Aug 2018 15:42:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:cc:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language; bh=AUFpXn1W9mdeE1h8VoE9q4OOz4FTRF3MU3g1hg74ZCk=; b=REH53ZSWnd+pOLu5BXS48JO9uPzhQIa5OudnxJZbvg6Cggs5aSmOaLqtbRP6sqt3WK TQ5jJyCeZ6fYF8u7YJ7+EU1uVyiw7BvCLGdhS7US9xehLg7pWwdPXVyyOTA6Jx2XxRyG mlf1KjrFBbKQ37PI9qjQ/gZh1ocg3Vj7hjVON9aTnXGuSZNhFbQhlYzMaNCp2hprk9AT YV9EQekr0XAJdFMYxM7D7wY+A7fqf7/9sYGhnMVSNXxaqhWlW+ZeNEwNXFxZ7Cc6K5bv 2KDh7uf9scNlyGrgttoaKyc70Yfrkb12f/uBiyctmTrBXrR9uSD0J49D8DOFtdalrxjP w7Zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=AUFpXn1W9mdeE1h8VoE9q4OOz4FTRF3MU3g1hg74ZCk=; b=V3kkJD9ea1D+g3w4R44Q/kRsvlz8QNCTQ2Knz68bgKbaikluI6mJciMuglL0I2mku3 uAOhMua0IgoR+rryaY2WG4gy4mm3NwAonGs3Y9c1tIFP8u5K+vYbL59cueUSOdKQXMn5 NG9VcyTtUQO6LJV1CwEqDNKckWZP/ryKpLVI1PSe96tOxoppIGgRgU65WcOIMCkhVcp7 yTSeU1GgKXjzCNHPx7Q3Rchximl7T7hE8jdrIZKcHC1hbPFnoZrDv2eXStINM7MoUyVy fm5TJl3q/6KCyg0BRvcKT1TmNCaJcoGKG0D85Aj04941+lGFroaSnWLLfj0SSCLTKqtK Z9Uw== X-Gm-Message-State: APzg51AQy2pMvMjLZvCWkuUUEkVMzX6YuFsIWK2dC7Xmli5i+RcY5tLG C8NtMAgoWmcY7w/xX1GDLYCebcPWxRg= X-Google-Smtp-Source: ANB0VdYhvIBGlmblOu3Jr98cTPfrBUv4eCCHkOYVYqs6CQIdDhZLMe8ceRc+NdHKl3ODlCOqxqdU1w== X-Received: by 2002:aca:3f41:: with SMTP id m62-v6mr635854oia.18.1535409737174; Mon, 27 Aug 2018 15:42:17 -0700 (PDT) Received: from ?IPv6:2601:2c2:800:300::ff24? ([2601:2c2:800:300::ff24]) by smtp.gmail.com with ESMTPSA id l195-v6sm814338oib.17.2018.08.27.15.42.16 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Aug 2018 15:42:16 -0700 (PDT) Subject: Re: Bhyve guest boot error, Buffer I/O error on dev vda1, logical block 0, lost async page write Cc: FreeBSD virtualization References: <114c2000-17ba-3cf6-1d4a-1354f42d8875@gmail.com> From: Rajil Saraswat Message-ID: <69322b42-dc78-e2b0-d92d-e70cc37924d9@gmail.com> Date: Mon, 27 Aug 2018 17:42:10 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 22:42:18 -0000 On 08/27/2018 11:50 AM, Pete Wright wrote: > > > On 8/26/18 10:07 PM, Sami Halabi wrote: >> Hi, >> IIRC pool shouldn't be above 90% used otherwise bad things happen.. >> but at >> 69%???? >> Can someone explain thid please? > > it's possible the filesystem was heavily fragmented?  seems suspicious > - did scrubbing your pool find any errors? > > -pete > This is how the pool looked like, |# zpool list vmpool NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT vmpool 372G 257G 115G - - 64% 69% 1.00x ONLINE - | Scrub did not show the errors. After deleting unused zvols, the guests started to work and pool looks like this, | | |# zpool list vmpool NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT vmpool   372G   197G   175G        -         -    56%    53%  1.00x  ONLINE  - | | | From owner-freebsd-virtualization@freebsd.org Tue Aug 28 03:27:50 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D449610A0381 for ; Tue, 28 Aug 2018 03:27:49 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D1B570269 for ; Tue, 28 Aug 2018 03:27:49 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id b19-v6so319941wme.3 for ; Mon, 27 Aug 2018 20:27:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=m1lmhXyLOLGpdz1i1WOSTAbi3sfWl98m+stg6SX/nrA=; b=r53UQXXoitOdcPFXgz02qUyG5tAOCnpTQQS/hZI4AQXK6Y9ff2ScreY33PoboLi4A+ 0yufY+9HmraaXbwDOUlN2R5iR3OZEMM0JzjRofZKeasVxSZLdR8xNYXzR0gDpztvM+cd 7RYMafzwUmOdFb2g6R4yX8mHUGKNXmMvXPNRJalmRW3gM4Oq9MiviVMV7l+bMtXPFNaD o+YocxejnzF6cqOCPWN9J6CMiAUJD4dkgWgaN/D3el3ENJm5y7aTng0m851n5nmK0v0g bObqZ4FIa1woasYjITG8lYrICtGFGDxM4hQ9lSDe9xoJ9Ii0aL6wgaNYOouw+Sooq+XW eBGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=m1lmhXyLOLGpdz1i1WOSTAbi3sfWl98m+stg6SX/nrA=; b=MQooVIdtpIFs8Ks+PUGxvfvcYq7tEehspi+pzPuiNkNWCBfdd04SFtIluT8WdYGJIA BLD/yik2gUNfaO9v1PmmtsTJyFeJHVZHopARZWddMHa5UOKTgwV5XuJylbRFkV13ElEe yUtv2mAVSSswsh59KHGJj4pGTnzZw/gqJWY/5cI6tX13yZmyT4LIDr5bhBogatojZfcz nHRd4s29aqfXLlUG3PdQxZYy9BoAxZBxYGyrrdJgxEupmATNEQ/avB5GX8k2LHINBcjK p6fAqC9BYZYzlL3fUBg5wHr/JXBRMVF1Dr2oalQ7wv6GykXn+dYn+EpG7mQCwsA2hgcB Or6w== X-Gm-Message-State: APzg51CCjP4xl3V3poC1bib3oZ9hncZ4GNx/9+ZvHWaJyjaFcpzLuPzJ yOCKGoIbkBkDXZkeoQJpV7TEmfrOP/QvdHR6xZo= X-Google-Smtp-Source: ANB0VdY3ls4MpuAZKNoEe46FMcJXRkWrQsv14hEePoJHo8Lp7Gdg9jNpVhHO0+H9BsVTYYAXXM3aN/9ZE2gfb34CWjk= X-Received: by 2002:a1c:6386:: with SMTP id x128-v6mr41051wmb.104.1535426867496; Mon, 27 Aug 2018 20:27:47 -0700 (PDT) MIME-Version: 1.0 References: <201808271519.w7RFJt5J011000@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201808271519.w7RFJt5J011000@pdx.rh.CN85.dnsmgr.net> From: Anish Date: Mon, 27 Aug 2018 20:27:36 -0700 Message-ID: Subject: Re: on bhyve statistics To: "Rodney W. Grimes" Cc: Fabian Freyer , "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 03:27:50 -0000 >> Currently, bhyve does not expose any of these statistics. All the stats > available through bhyvectl --get-stats seem to be coming from the VMM, > not from the userspace emulation. >That is correct, byhvectl is a diagnostics tool for getting information from the kernel/vmm module. bhyvectl provide stats related to processor vmx/svm from vmm.ko and is the first thing you want to run for performance regression. It will be nice to include it as part of bhyve perf tool/dashboard that you are intended to build. -Anish On Mon, Aug 27, 2018 at 8:20 AM Rodney W. Grimes < freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > Hi list, > > > > I'm currently looking at getting the libvirt prometheus exporter[1] to > > work with libvirt+bhyve. In its current state this doesn't work because > > at least one of the API calls exposed by libvirt isn't implemented by > > the libvirt bhyve driver - so I started looking at implementing it. > > > > The first API call in question is virDomainBlockStats[2], which returns > > statistics (number of read and written bytes and ops, respectively). > > > > Currently, bhyve does not expose any of these statistics. All the stats > > available through bhyvectl --get-stats seem to be coming from the VMM, > > not from the userspace emulation. > > That is correct, byhvectl is a diagnostics tool for getting > information from the kernel/vmm module. > > > OTOH, I did see that there are *some* > > stats being collected in bhyverun.c (see struct bhyvestats {...} > > stats;). I can't see how these are exposed though - a grep of /usr/src > > turned up no other uses. Which brings me to the following questions: > > > > - are the stats in struct bhyvestats {...} stats exposed or used in any > > non-obvious way? > > Not that I am aware of. > > > - architecturally, what would be the best ways to get stats out of the > > user-space emulations? Off of the top of my head, I could think of the > > following possibilities: > > - prometheus exporter > > - having some socket or pipe to request them > > - DTrace probes > > > > I wouldn't mind implementing any of the above, and so would like to know > > which of these (or other options) would be the most acceptable, and > > would appreciate some guidance. > > I differ to others on what may be the best way to do this. > > > CC'ing novel@ for the libvirt side, and grehan@ for the architectural > > bhyve questions. > > You should replace @grehan with @jhb,@tychon as Peter has moved on, > and John and Tycho are now the bhyve maintainers. I was going to > add them, and remove Peter, but I see no cc: anyway, so I am sure > that they are on the virtualization list though. > > > Fabian > > > > [1] https://github.com/kumina/libvirt_exporter > > [2] > https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockStats > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > > -- > Rod Grimes > rgrimes@freebsd.org > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@freebsd.org Tue Aug 28 04:56:17 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78CA310A1FF2 for ; Tue, 28 Aug 2018 04:56:17 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DCC7372663 for ; Tue, 28 Aug 2018 04:56:16 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id w7S4u8Ah063076 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 27 Aug 2018 21:56:09 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id w7S4u8MV063075; Mon, 27 Aug 2018 21:56:08 -0700 (PDT) (envelope-from jmg) Date: Mon, 27 Aug 2018 21:56:08 -0700 From: John-Mark Gurney To: Rajil Saraswat Cc: FreeBSD virtualization Subject: Re: Bhyve guest boot error, Buffer I/O error on dev vda1, logical block 0, lost async page write Message-ID: <20180828045608.GQ45503@funkthat.com> References: <114c2000-17ba-3cf6-1d4a-1354f42d8875@gmail.com> <69322b42-dc78-e2b0-d92d-e70cc37924d9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <69322b42-dc78-e2b0-d92d-e70cc37924d9@gmail.com> X-Operating-System: FreeBSD 11.0-RELEASE-p7 amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Mon, 27 Aug 2018 21:56:09 -0700 (PDT) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 04:56:17 -0000 Rajil Saraswat wrote this message on Mon, Aug 27, 2018 at 17:42 -0500: > On 08/27/2018 11:50 AM, Pete Wright wrote: > > > > > > On 8/26/18 10:07 PM, Sami Halabi wrote: > >> Hi, > >> IIRC pool shouldn't be above 90% used otherwise bad things happen.. > >> but at > >> 69%???? > >> Can someone explain thid please? > > > > it's possible the filesystem was heavily fragmented?  seems suspicious > > - did scrubbing your pool find any errors? > > > > -pete > > > > This is how the pool looked like, > > |# zpool list vmpool NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP > HEALTH ALTROOT vmpool 372G 257G 115G - - 64% 69% 1.00x ONLINE - | > > Scrub did not show the errors. After deleting unused zvols, the guests > started to work and pool looks like this, > | > | > > |# zpool list vmpool > NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  > HEALTH  ALTROOT > vmpool   372G   197G   175G        -         -    56%    53%  1.00x  > ONLINE  - There are a couple things that can happen.. you could have a quota or refquota that prevents a dataset/zdev, or it's descendants from growing larger, or you could have refreservation or reservation set which limits the space that other datasets can consume.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@freebsd.org Tue Aug 28 10:32:09 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4311B1087702 for ; Tue, 28 Aug 2018 10:32:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D4FF27D416 for ; Tue, 28 Aug 2018 10:32:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 968531087700; Tue, 28 Aug 2018 10:32:08 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 852FA10876FF for ; Tue, 28 Aug 2018 10:32:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23CA57D410 for ; Tue, 28 Aug 2018 10:32:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 826B31FF2A for ; Tue, 28 Aug 2018 10:32:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7SAW7ZU081281 for ; Tue, 28 Aug 2018 10:32:07 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7SAW7S7081280 for virtualization@FreeBSD.org; Tue, 28 Aug 2018 10:32:07 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230970] [bhyve] bridge interface slow downs bandwith speed Date: Tue, 28 Aug 2018 10:32:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: component short_desc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 10:32:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230970 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Component|standards |misc Summary|bridge interface slow downs |[bhyve] bridge interface |bandwith speed |slow downs bandwith speed Assignee|standards@FreeBSD.org |virtualization@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Aug 28 13:37:21 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C15C108C936 for ; Tue, 28 Aug 2018 13:37:21 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0815A842B7 for ; Tue, 28 Aug 2018 13:37:20 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7SDbB3G015574; Tue, 28 Aug 2018 06:37:11 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7SDbAEe015573; Tue, 28 Aug 2018 06:37:11 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808281337.w7SDbAEe015573@pdx.rh.CN85.dnsmgr.net> Subject: Re: on bhyve statistics In-Reply-To: To: Anish Date: Tue, 28 Aug 2018 06:37:10 -0700 (PDT) CC: Fabian Freyer , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 13:37:21 -0000 > >> Currently, bhyve does not expose any of these statistics. All the stats > > available through bhyvectl --get-stats seem to be coming from the VMM, > > not from the userspace emulation. > > >That is correct, byhvectl is a diagnostics tool for getting > information from the kernel/vmm module. > > bhyvectl provide stats related to processor vmx/svm from vmm.ko and is the > first thing you want to run for performance regression. It will be nice to > include it as part of bhyve perf tool/dashboard that you are intended to > build. >From conversations with Peter Grehan he expressed that bhyvectl is purely a diagnostics tool that should not be depended on by any other tools. If you want to do similiar things you should program to the libvmmapi interface, not bhyvectl. > -Anish > > On Mon, Aug 27, 2018 at 8:20 AM Rodney W. Grimes < > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > Hi list, > > > > > > I'm currently looking at getting the libvirt prometheus exporter[1] to > > > work with libvirt+bhyve. In its current state this doesn't work because > > > at least one of the API calls exposed by libvirt isn't implemented by > > > the libvirt bhyve driver - so I started looking at implementing it. > > > > > > The first API call in question is virDomainBlockStats[2], which returns > > > statistics (number of read and written bytes and ops, respectively). > > > > > > Currently, bhyve does not expose any of these statistics. All the stats > > > available through bhyvectl --get-stats seem to be coming from the VMM, > > > not from the userspace emulation. > > > > That is correct, byhvectl is a diagnostics tool for getting > > information from the kernel/vmm module. > > > > > OTOH, I did see that there are *some* > > > stats being collected in bhyverun.c (see struct bhyvestats {...} > > > stats;). I can't see how these are exposed though - a grep of /usr/src > > > turned up no other uses. Which brings me to the following questions: > > > > > > - are the stats in struct bhyvestats {...} stats exposed or used in any > > > non-obvious way? > > > > Not that I am aware of. > > > > > - architecturally, what would be the best ways to get stats out of the > > > user-space emulations? Off of the top of my head, I could think of the > > > following possibilities: > > > - prometheus exporter > > > - having some socket or pipe to request them > > > - DTrace probes > > > > > > I wouldn't mind implementing any of the above, and so would like to know > > > which of these (or other options) would be the most acceptable, and > > > would appreciate some guidance. > > > > I differ to others on what may be the best way to do this. > > > > > CC'ing novel@ for the libvirt side, and grehan@ for the architectural > > > bhyve questions. > > > > You should replace @grehan with @jhb,@tychon as Peter has moved on, > > and John and Tycho are now the bhyve maintainers. I was going to > > add them, and remove Peter, but I see no cc: anyway, so I am sure > > that they are on the virtualization list though. > > > > > Fabian > > > > > > [1] https://github.com/kumina/libvirt_exporter > > > [2] > > https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockStats > > -- > > Rod Grimes > > rgrimes@freebsd.org -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Tue Aug 28 13:44:17 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4779E108CC9D for ; Tue, 28 Aug 2018 13:44:17 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A13CE8483D for ; Tue, 28 Aug 2018 13:44:16 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lf1-x12a.google.com with SMTP id g6-v6so1410119lfb.11 for ; Tue, 28 Aug 2018 06:44:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=uu/iydn6by3NEoo2Bi8S7YxyRSrD0KRR/ayyGa2udlY=; b=tXSFiV2S0FARD1hiG780r6ETV3PVp+845WEpcfI48lBIknJKZKYPNPVZwBWHkGq6Da 6wkOFjuS01J2Jb5hFvXX1OTWUWNcV1OjTxIt2lab2dViVXd2on+LxiehSczsy7m1QdT6 dIcWTy+98aqMGK73kGgkcvexghkR3vKNuQjjZWJCZHHiKJ5yrzd6iPMajckIqsltnYYM hbWxrZOktfy0rlDaq0JU0l3aHdBYTlz1E2qxEZBbW20fmw17LF6FiFeSzpi6/BTLAO+6 TK7PYrU+84OTapgi7ltCOSkwVA+4aJSuL9fkzmJB7cVt/ohlns59STWkS4/ktKypl2bn W5tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=uu/iydn6by3NEoo2Bi8S7YxyRSrD0KRR/ayyGa2udlY=; b=giHzcwlRkGLg89B2FPVCNqkkqLyT38hxmw1oMEwuhNrsnMJBBM+Zk/inz76XIygm0U 6JYStFugcRFIQnpDubtSS0crZ02mUOxxRMkEH4S0yiwQPVF4TX+avJ0SaoRUwDX0oghI KY6xLsnJAAqqhZLZUT/X8hqh7Lm/Kl1CFYetue18UijR5D2tb9hceGOj/ZXNkszrTtNx iC39XssP0fh/oesonkz0cYXlK0Z+5SXOPkFmU95p3Y3L1+iYcxuP/2CZgi6G2jG8S483 KWa6OE42MXIMtppHtgS58k+shCZekg6h6AekYut9H2GoYg/fbHRJyVdCHSANcOw9lQd1 8nsw== X-Gm-Message-State: APzg51A6Cs7VqjqK1e3YEtpeK7/EONhgvQU0FQbtWV8xX1t0b3caPbHQ 2/e7eiuGqwmtc+8JkTC5vgVZkMZP2bIa65inM8IrDA== X-Google-Smtp-Source: ANB0VdZZJh/IMqtwKnorJY1lCaahar1+R06k7eThj0wOlegpfkeh13gs7zPDzO5jAts7bfDpWxWDPstJNaxlPBbiO7o= X-Received: by 2002:a19:54c2:: with SMTP id b63-v6mr1406634lfl.128.1535463855305; Tue, 28 Aug 2018 06:44:15 -0700 (PDT) MIME-Version: 1.0 References: <201808281337.w7SDbAEe015573@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201808281337.w7SDbAEe015573@pdx.rh.CN85.dnsmgr.net> Reply-To: araujo@freebsd.org From: Marcelo Araujo Date: Tue, 28 Aug 2018 21:44:02 +0800 Message-ID: Subject: Re: on bhyve statistics To: "Rodney W. Grimes" Cc: Anish , freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 13:44:17 -0000 On Tue, Aug 28, 2018, 9:38 PM Rodney W. Grimes < freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > >> Currently, bhyve does not expose any of these statistics. All the > stats > > > available through bhyvectl --get-stats seem to be coming from the VMM, > > > not from the userspace emulation. > > > > >That is correct, byhvectl is a diagnostics tool for getting > > information from the kernel/vmm module. > > > > bhyvectl provide stats related to processor vmx/svm from vmm.ko and is > the > > first thing you want to run for performance regression. It will be nice > to > > include it as part of bhyve perf tool/dashboard that you are intended to > > build. > > From conversations with Peter Grehan he expressed that bhyvectl is > purely a diagnostics tool that should not be depended on by any > other tools. > > If you want to do similiar things you should program to the libvmmapi > interface, not bhyvectl. > The libvmmapi is more an internal library for usage with bhyvectl and bhyveload than for other purposes, of course it won't stop anybody to extend it to achieve other goals. > > -Anish > > > > On Mon, Aug 27, 2018 at 8:20 AM Rodney W. Grimes < > > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > Hi list, > > > > > > > > I'm currently looking at getting the libvirt prometheus exporter[1] > to > > > > work with libvirt+bhyve. In its current state this doesn't work > because > > > > at least one of the API calls exposed by libvirt isn't implemented by > > > > the libvirt bhyve driver - so I started looking at implementing it. > > > > > > > > The first API call in question is virDomainBlockStats[2], which > returns > > > > statistics (number of read and written bytes and ops, respectively). > > > > > > > > Currently, bhyve does not expose any of these statistics. All the > stats > > > > available through bhyvectl --get-stats seem to be coming from the > VMM, > > > > not from the userspace emulation. > > > > > > That is correct, byhvectl is a diagnostics tool for getting > > > information from the kernel/vmm module. > > > > > > > OTOH, I did see that there are *some* > > > > stats being collected in bhyverun.c (see struct bhyvestats {...} > > > > stats;). I can't see how these are exposed though - a grep of > /usr/src > > > > turned up no other uses. Which brings me to the following questions: > > > > > > > > - are the stats in struct bhyvestats {...} stats exposed or used in > any > > > > non-obvious way? > > > > > > Not that I am aware of. > > > > > > > - architecturally, what would be the best ways to get stats out of > the > > > > user-space emulations? Off of the top of my head, I could think of > the > > > > following possibilities: > > > > - prometheus exporter > > > > - having some socket or pipe to request them > > > > - DTrace probes > > > > > > > > I wouldn't mind implementing any of the above, and so would like to > know > > > > which of these (or other options) would be the most acceptable, and > > > > would appreciate some guidance. > > > > > > I differ to others on what may be the best way to do this. > > > > > > > CC'ing novel@ for the libvirt side, and grehan@ for the > architectural > > > > bhyve questions. > > > > > > You should replace @grehan with @jhb,@tychon as Peter has moved on, > > > and John and Tycho are now the bhyve maintainers. I was going to > > > add them, and remove Peter, but I see no cc: anyway, so I am sure > > > that they are on the virtualization list though. > > > > > > > Fabian > > > > > > > > [1] https://github.com/kumina/libvirt_exporter > > > > [2] > > > > https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockStats > > > -- > > > Rod Grimes > > > rgrimes@freebsd.org > > -- > Rod Grimes > rgrimes@freebsd.org > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@freebsd.org Tue Aug 28 13:54:53 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2DC92108D292 for ; Tue, 28 Aug 2018 13:54:53 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B642085272; Tue, 28 Aug 2018 13:54:52 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7SDsp9O015652; Tue, 28 Aug 2018 06:54:51 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7SDspUw015651; Tue, 28 Aug 2018 06:54:51 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808281354.w7SDspUw015651@pdx.rh.CN85.dnsmgr.net> Subject: Re: on bhyve statistics In-Reply-To: To: araujo@freebsd.org Date: Tue, 28 Aug 2018 06:54:51 -0700 (PDT) CC: Anish , freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 13:54:53 -0000 > On Tue, Aug 28, 2018, 9:38 PM Rodney W. Grimes < > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > >> Currently, bhyve does not expose any of these statistics. All the > > stats > > > > available through bhyvectl --get-stats seem to be coming from the VMM, > > > > not from the userspace emulation. > > > > > > >That is correct, byhvectl is a diagnostics tool for getting > > > information from the kernel/vmm module. > > > > > > bhyvectl provide stats related to processor vmx/svm from vmm.ko and is > > the > > > first thing you want to run for performance regression. It will be nice > > to > > > include it as part of bhyve perf tool/dashboard that you are intended to > > > build. > > > > From conversations with Peter Grehan he expressed that bhyvectl is > > purely a diagnostics tool that should not be depended on by any > > other tools. > > > > If you want to do similiar things you should program to the libvmmapi > > interface, not bhyvectl. > > > > The libvmmapi is more an internal library for usage with bhyvectl and > bhyveload than for other purposes, of course it won't stop anybody to > extend it to achieve other goals. That is not my understanding at all, libvmmapi is the external and stable/maintained interface to vmm.ko, it is not at all strickly for use by bhyvectl or bhyveload or bhyve. And as such this "api" should be very carefully maintained as it needs to be caried forward for ever. > > > -Anish > > > > > > On Mon, Aug 27, 2018 at 8:20 AM Rodney W. Grimes < > > > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > > > Hi list, > > > > > > > > > > I'm currently looking at getting the libvirt prometheus exporter[1] > > to > > > > > work with libvirt+bhyve. In its current state this doesn't work > > because > > > > > at least one of the API calls exposed by libvirt isn't implemented by > > > > > the libvirt bhyve driver - so I started looking at implementing it. > > > > > > > > > > The first API call in question is virDomainBlockStats[2], which > > returns > > > > > statistics (number of read and written bytes and ops, respectively). > > > > > > > > > > Currently, bhyve does not expose any of these statistics. All the > > stats > > > > > available through bhyvectl --get-stats seem to be coming from the > > VMM, > > > > > not from the userspace emulation. > > > > > > > > That is correct, byhvectl is a diagnostics tool for getting > > > > information from the kernel/vmm module. > > > > > > > > > OTOH, I did see that there are *some* > > > > > stats being collected in bhyverun.c (see struct bhyvestats {...} > > > > > stats;). I can't see how these are exposed though - a grep of > > /usr/src > > > > > turned up no other uses. Which brings me to the following questions: > > > > > > > > > > - are the stats in struct bhyvestats {...} stats exposed or used in > > any > > > > > non-obvious way? > > > > > > > > Not that I am aware of. > > > > > > > > > - architecturally, what would be the best ways to get stats out of > > the > > > > > user-space emulations? Off of the top of my head, I could think of > > the > > > > > following possibilities: > > > > > - prometheus exporter > > > > > - having some socket or pipe to request them > > > > > - DTrace probes > > > > > > > > > > I wouldn't mind implementing any of the above, and so would like to > > know > > > > > which of these (or other options) would be the most acceptable, and > > > > > would appreciate some guidance. > > > > > > > > I differ to others on what may be the best way to do this. > > > > > > > > > CC'ing novel@ for the libvirt side, and grehan@ for the > > architectural > > > > > bhyve questions. > > > > > > > > You should replace @grehan with @jhb,@tychon as Peter has moved on, > > > > and John and Tycho are now the bhyve maintainers. I was going to > > > > add them, and remove Peter, but I see no cc: anyway, so I am sure > > > > that they are on the virtualization list though. > > > > > > > > > Fabian > > > > > > > > > > [1] https://github.com/kumina/libvirt_exporter > > > > > [2] > > > > > > https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockStats > > > > -- > > > > Rod Grimes > > > > rgrimes@freebsd.org > > > > -- > > Rod Grimes > > rgrimes@freebsd.org > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to " > > freebsd-virtualization-unsubscribe@freebsd.org" > > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Tue Aug 28 14:21:28 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E354108DD7A for ; Tue, 28 Aug 2018 14:21:28 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com [IPv6:2a00:1450:4864:20::129]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 76E3A8611C; Tue, 28 Aug 2018 14:21:27 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lf1-x129.google.com with SMTP id e23-v6so1512432lfc.13; Tue, 28 Aug 2018 07:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=76qVTlSc/PeAq8pDSX1z9dxRNACKvOxWOeffyXuQ68M=; b=hE20d26qTILDdYaIi0MUkcTv0N4KNf/Z2UAPjdf2ALuo7V5VQixXujeh0qAonflIDV Am9l0iTG3xbp9uGHAXK0JwWTbTrGOTiav+ahPn9Y9J2T+tea3hHb1LWQhu4Cd3nuuwIl gOE2LYOR4TjFCTebusdzr2b4mAM6CSq3EHJ8Sd8D7iE416S/G7t+aS2IJS+6VSVhP7eC V/6FUe63mfZ0MT/N8sTFtYfxsFMURhIHj2F1R03uMe2cjxlSIeV/X/LAOnilLwDgWbG9 lE8sX7w/R3y46pumo3pVQmmHEoyokfduvKBKQuZqFWOL5c6bupRSBmeUHrg1sNBVtzJN gXVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=76qVTlSc/PeAq8pDSX1z9dxRNACKvOxWOeffyXuQ68M=; b=pijn/2huM0wIPFqBOXxOyjO8i7hKoEPGy6knG4o2buv8KtBBIsM6G0OiXNG2zDWgVn 4RJwDT+IPfwEP2hioVobbUbVzRO3S9LVfdkob0+y3wfiKXIAfwlFXm5hX6vSujc794yZ oMF76Gh/m4QSCfd+Yh02z0kAfhUPAMDSqKCeufPbl3V+x0qpc4CKaWdoLDG0aVD5Ovw8 2P9I8welVX5aHErPLX7dcEZBLuLPZpPtpU8p5YA/fxW1Ade0y+oMLp74BMjlZrTdbzIl OlQIWwo4gZ0nKT9TstpgdzI5nZpntgLhWRTBOTHoFsUnkTX7k0zG0rCX878hOvHMBQJc 3NUg== X-Gm-Message-State: APzg51BcwjUANKadYEMYZFyuOPrIFXb//7b2ez/APzclHGH92Pqlk62F jQxwxkTNvShcYhaNWULF6oaRfNXd+PQy49YCGgA= X-Google-Smtp-Source: ANB0VdazgjC0awmlFVD9E72Q/fVTIwAVL+ZErN99BWPnJ5BL70WTDXAh2eyom1Cxf3W83Mt21/PGqZJWRBiMaAqpGbE= X-Received: by 2002:a19:54c2:: with SMTP id b63-v6mr1513513lfl.128.1535466085443; Tue, 28 Aug 2018 07:21:25 -0700 (PDT) MIME-Version: 1.0 References: <201808281354.w7SDspUw015651@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201808281354.w7SDspUw015651@pdx.rh.CN85.dnsmgr.net> Reply-To: araujo@freebsd.org From: Marcelo Araujo Date: Tue, 28 Aug 2018 22:21:12 +0800 Message-ID: Subject: Re: on bhyve statistics To: "Rodney W. Grimes" Cc: Marcelo Araujo , Anish , freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 14:21:28 -0000 On Tue, Aug 28, 2018, 9:54 PM Rodney W. Grimes < freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > On Tue, Aug 28, 2018, 9:38 PM Rodney W. Grimes < > > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > >> Currently, bhyve does not expose any of these statistics. All the > > > stats > > > > > available through bhyvectl --get-stats seem to be coming from the > VMM, > > > > > not from the userspace emulation. > > > > > > > > >That is correct, byhvectl is a diagnostics tool for getting > > > > information from the kernel/vmm module. > > > > > > > > bhyvectl provide stats related to processor vmx/svm from vmm.ko and > is > > > the > > > > first thing you want to run for performance regression. It will be > nice > > > to > > > > include it as part of bhyve perf tool/dashboard that you are > intended to > > > > build. > > > > > > From conversations with Peter Grehan he expressed that bhyvectl is > > > purely a diagnostics tool that should not be depended on by any > > > other tools. > > > > > > If you want to do similiar things you should program to the libvmmapi > > > interface, not bhyvectl. > > > > > > > The libvmmapi is more an internal library for usage with bhyvectl and > > bhyveload than for other purposes, of course it won't stop anybody to > > extend it to achieve other goals. > > That is not my understanding at all, libvmmapi is the external and > stable/maintained interface to vmm.ko, it is not at all strickly > for use by bhyvectl or bhyveload or bhyve. > You need something between userland and vmm. > > And as such this "api" should be very carefully maintained as > it needs to be caried forward for ever. > I won't call it as an API provider as such you cannot do all operations from creating to launch a guest! > > > > -Anish > > > > > > > > On Mon, Aug 27, 2018 at 8:20 AM Rodney W. Grimes < > > > > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > > > > > Hi list, > > > > > > > > > > > > I'm currently looking at getting the libvirt prometheus > exporter[1] > > > to > > > > > > work with libvirt+bhyve. In its current state this doesn't work > > > because > > > > > > at least one of the API calls exposed by libvirt isn't > implemented by > > > > > > the libvirt bhyve driver - so I started looking at implementing > it. > > > > > > > > > > > > The first API call in question is virDomainBlockStats[2], which > > > returns > > > > > > statistics (number of read and written bytes and ops, > respectively). > > > > > > > > > > > > Currently, bhyve does not expose any of these statistics. All the > > > stats > > > > > > available through bhyvectl --get-stats seem to be coming from the > > > VMM, > > > > > > not from the userspace emulation. > > > > > > > > > > That is correct, byhvectl is a diagnostics tool for getting > > > > > information from the kernel/vmm module. > > > > > > > > > > > OTOH, I did see that there are *some* > > > > > > stats being collected in bhyverun.c (see struct bhyvestats {...} > > > > > > stats;). I can't see how these are exposed though - a grep of > > > /usr/src > > > > > > turned up no other uses. Which brings me to the following > questions: > > > > > > > > > > > > - are the stats in struct bhyvestats {...} stats exposed or used > in > > > any > > > > > > non-obvious way? > > > > > > > > > > Not that I am aware of. > > > > > > > > > > > - architecturally, what would be the best ways to get stats out > of > > > the > > > > > > user-space emulations? Off of the top of my head, I could > think of > > > the > > > > > > following possibilities: > > > > > > - prometheus exporter > > > > > > - having some socket or pipe to request them > > > > > > - DTrace probes > > > > > > > > > > > > I wouldn't mind implementing any of the above, and so would like > to > > > know > > > > > > which of these (or other options) would be the most acceptable, > and > > > > > > would appreciate some guidance. > > > > > > > > > > I differ to others on what may be the best way to do this. > > > > > > > > > > > CC'ing novel@ for the libvirt side, and grehan@ for the > > > architectural > > > > > > bhyve questions. > > > > > > > > > > You should replace @grehan with @jhb,@tychon as Peter has moved on, > > > > > and John and Tycho are now the bhyve maintainers. I was going to > > > > > add them, and remove Peter, but I see no cc: anyway, so I am sure > > > > > that they are on the virtualization list though. > > > > > > > > > > > Fabian > > > > > > > > > > > > [1] https://github.com/kumina/libvirt_exporter > > > > > > [2] > > > > > > > > > https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockStats > > > > > -- > > > > > Rod Grimes > > > > > rgrimes@freebsd.org > > > > > > -- > > > Rod Grimes > > > rgrimes@freebsd.org > > > _______________________________________________ > > > freebsd-virtualization@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > > To unsubscribe, send any mail to " > > > freebsd-virtualization-unsubscribe@freebsd.org" > > > > > -- > Rod Grimes > rgrimes@freebsd.org > From owner-freebsd-virtualization@freebsd.org Tue Aug 28 14:50:41 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B30CA108EB9D for ; Tue, 28 Aug 2018 14:50:41 +0000 (UTC) (envelope-from fabian.freyer@physik.tu-berlin.de) Received: from mail.physik.tu-berlin.de (mail.physik-pool.tu-berlin.de [130.149.50.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 536CC875E8; Tue, 28 Aug 2018 14:50:40 +0000 (UTC) (envelope-from fabian.freyer@physik.tu-berlin.de) Received: from [192.168.0.114] (unknown [130.149.50.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.physik.tu-berlin.de (Postfix) with ESMTPSA id 9A5EA61FA3; Tue, 28 Aug 2018 14:50:32 +0000 (UTC) Subject: Re: on bhyve statistics To: "Rodney W. Grimes" , Anish Cc: "freebsd-virtualization@freebsd.org" , Marcelo Araujo References: <201808281337.w7SDbAEe015573@pdx.rh.CN85.dnsmgr.net> From: Fabian Freyer Message-ID: <79435a58-9591-b55a-adca-81e037e904ff@physik.tu-berlin.de> Date: Tue, 28 Aug 2018 16:50:17 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 In-Reply-To: <201808281337.w7SDbAEe015573@pdx.rh.CN85.dnsmgr.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 14:50:41 -0000 On 8/28/18 3:37 PM, Rodney W. Grimes wrote: >>>> Currently, bhyve does not expose any of these statistics. All the stats >>> available through bhyvectl --get-stats seem to be coming from the VMM, >>> not from the userspace emulation. >> >>> That is correct, byhvectl is a diagnostics tool for getting >> information from the kernel/vmm module. >> >> bhyvectl provide stats related to processor vmx/svm from vmm.ko and is the >> first thing you want to run for performance regression. It will be nice to >> include it as part of bhyve perf tool/dashboard that you are intended to >> build. > > From conversations with Peter Grehan he expressed that bhyvectl is > purely a diagnostics tool that should not be depended on by any > other tools. > > If you want to do similiar things you should program to the libvmmapi > interface, not bhyvectl. For context, this is *not* what I'm aiming to do. While bhyvectl just prints statistics exposed by the VMM through libvmmapi, I'm specifically asking about instrumenting the userland part, bhyve. Also, libvmmapi does have downstream consumers that are not bhyve{,ctl,load}, e.g. grub2-bhyve, bhyve-multiboot[1], maybe even xhyve[2] - for performance reasons, I'd prefer scraping metrics through libvmmapi over shelling out to bhyvectl. [1] https://github.com/fubarnetes/bhyve-multiboot [2] https://github.com/mist64/xhyve From owner-freebsd-virtualization@freebsd.org Tue Aug 28 15:31:37 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E125E108F8A1 for ; Tue, 28 Aug 2018 15:31:37 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70DC389EDF for ; Tue, 28 Aug 2018 15:31:37 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7SFVXNN016213; Tue, 28 Aug 2018 08:31:33 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7SFVWfV016212; Tue, 28 Aug 2018 08:31:32 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808281531.w7SFVWfV016212@pdx.rh.CN85.dnsmgr.net> Subject: Re: on bhyve statistics In-Reply-To: <79435a58-9591-b55a-adca-81e037e904ff@physik.tu-berlin.de> To: Fabian Freyer Date: Tue, 28 Aug 2018 08:31:32 -0700 (PDT) CC: Anish , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 15:31:38 -0000 > On 8/28/18 3:37 PM, Rodney W. Grimes wrote: > >>>> Currently, bhyve does not expose any of these statistics. All the stats > >>> available through bhyvectl --get-stats seem to be coming from the VMM, > >>> not from the userspace emulation. > >> > >>> That is correct, byhvectl is a diagnostics tool for getting > >> information from the kernel/vmm module. > >> > >> bhyvectl provide stats related to processor vmx/svm from vmm.ko and is the > >> first thing you want to run for performance regression. It will be nice to > >> include it as part of bhyve perf tool/dashboard that you are intended to > >> build. > > > > From conversations with Peter Grehan he expressed that bhyvectl is > > purely a diagnostics tool that should not be depended on by any > > other tools. > > > > If you want to do similiar things you should program to the libvmmapi > > interface, not bhyvectl. > > For context, this is *not* what I'm aiming to do. While bhyvectl just > prints statistics exposed by the VMM through libvmmapi, I'm specifically > asking about instrumenting the userland part, bhyve. > > Also, libvmmapi does have downstream consumers that are not > bhyve{,ctl,load}, e.g. grub2-bhyve, bhyve-multiboot[1], maybe even > xhyve[2] - for performance reasons, I'd prefer scraping metrics through > libvmmapi over shelling out to bhyvectl. Yes, there are other consumers of libvmmapi, why I stressed that it is the api to interface with vmm.ko. It might even make since to have the userland bhyve(8) maintain statistics in libvmmapi for exactly these reasons. > [1] https://github.com/fubarnetes/bhyve-multiboot > [2] https://github.com/mist64/xhyve -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Wed Aug 29 15:51:13 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B28E108DA05 for ; Wed, 29 Aug 2018 15:51:13 +0000 (UTC) (envelope-from vans9@yandex.ru) Received: from forward24o.cmail.yandex.net (forward24o.cmail.yandex.net [IPv6:2a02:6b8:0:1a72::254]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9731772436 for ; Wed, 29 Aug 2018 15:51:12 +0000 (UTC) (envelope-from vans9@yandex.ru) Received: from mxback7j.mail.yandex.net (mxback7j.mail.yandex.net [IPv6:2a02:6b8:0:1619::110]) by forward24o.cmail.yandex.net (Yandex) with ESMTP id B268621E40 for ; Wed, 29 Aug 2018 18:51:00 +0300 (MSK) Received: from localhost (localhost [::1]) by mxback7j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id L0eFAsSjwG-oxHSqPvu; Wed, 29 Aug 2018 18:50:59 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1535557859; bh=ZlOEIRrfbsyjcW9WCbYLioMeofQZUAnU2kPZRpmn5rY=; h=From:To:Subject:Date:Message-Id; b=eZWah4HBW3AnN8V8Ka65gdghEozaO2SyflDKi1BMW0BmrjqbkvsHHxNLWcNjUkapA ehka94lSagx/omy68UaFbjTmiHhq/Atm1o0/GU0e0UievOdNsrkKD8JafPPoT8WokZ NPQ8KthuSJJIL76QOioJb+7vU5B3x6elcuwB4nDg= Authentication-Results: mxback7j.mail.yandex.net; dkim=pass header.i=@yandex.ru Received: by myt4-7b9f23a5698b.qloud-c.yandex.net with HTTP; Wed, 29 Aug 2018 18:50:59 +0300 From: N V Envelope-From: vans9@yandex.ru To: freebsd-virtualization@freebsd.org Subject: Re: Possible bug: 11.2-RELEASE guest with vtnet and PF MIME-Version: 1.0 X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Wed, 29 Aug 2018 18:50:59 +0300 Message-Id: <1222821535557859@myt4-7b9f23a5698b.qloud-c.yandex.net> Content-Transfer-Encoding: 7bit Content-Type: text/plain X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2018 15:51:13 -0000 Hello. I confirm this with 11.2-RELEASE-p2 and vtnet interface. # ifconfig vtnet0: flags=8843 metric 0 mtu 1500 options=6c07bb ether ... hwaddr ... inet6 ....%vtnet0 prefixlen 64 scopeid 0x1 inet6 .... prefixlen 64 autoconf inet ... netmask 0xfffffe00 broadcast ... nd6 options=23 media: Ethernet 10Gbase-T status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 groups: lo Regards, Vans From owner-freebsd-virtualization@freebsd.org Wed Aug 29 20:12:19 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE2EC1094E05 for ; Wed, 29 Aug 2018 20:12:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6908A7FD08 for ; Wed, 29 Aug 2018 20:12:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 225D41094E04; Wed, 29 Aug 2018 20:12:19 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09A691094E03 for ; Wed, 29 Aug 2018 20:12:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A9D2B7FCFF for ; Wed, 29 Aug 2018 20:12:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id ECC4711BF6 for ; Wed, 29 Aug 2018 20:12:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7TKCH7J098067 for ; Wed, 29 Aug 2018 20:12:17 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7TKCHV0098066 for virtualization@FreeBSD.org; Wed, 29 Aug 2018 20:12:17 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 229167] [Hyper-V] [Jun 19, 2018] Recently FreeBSD VM panics during boot-up, especially with Mellanox VF configured Date: Wed, 29 Aug 2018 20:12:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: version Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2018 20:12:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229167 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Version|CURRENT |11.2-STABLE --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Sep 1 14:10:20 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5F1C4FF07F3 for ; Sat, 1 Sep 2018 14:10:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id EC5738D109 for ; Sat, 1 Sep 2018 14:10:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id AFBCBFF07EF; Sat, 1 Sep 2018 14:10:19 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E703FF07EE for ; Sat, 1 Sep 2018 14:10:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41E4A8D105 for ; Sat, 1 Sep 2018 14:10:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 87CA02EB41 for ; Sat, 1 Sep 2018 14:10:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w81EAIom085002 for ; Sat, 1 Sep 2018 14:10:18 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w81EAIC5085001 for virtualization@FreeBSD.org; Sat, 1 Sep 2018 14:10:18 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230402] With buildworld, the system can not use swap Date: Sat, 01 Sep 2018 14:10:18 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: chris@crvintel.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Sep 2018 14:10:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230402 --- Comment #10 from chris@crvintel.com --- I to am having this problem. I am trying to buildworld on a RPi 3 B+ (using FreeBSD Current with 1G of swap space and the build fails during make of the clang source. I notice too that in my dmesg log that I am getting: warning: total configured swap (1048576 pages) exceeds maximum recommended amount (924056 pages). I have been unable to find any useful documentation on kern.maxswzone. >From my own observations the build does not seem to use up much swap space = as the build fails. The buildworld on my Rpi 1 B+ with 512M of swap space works fine, and there= is no reference in dmesg about maximum recommended swap space being exceeded. Any support appreciated. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Sep 1 15:22:37 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01E99FF266B for ; Sat, 1 Sep 2018 15:22:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 984B170177 for ; Sat, 1 Sep 2018 15:22:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 585B2FF2669; Sat, 1 Sep 2018 15:22:36 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 366D2FF2667 for ; Sat, 1 Sep 2018 15:22:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D124B70174 for ; Sat, 1 Sep 2018 15:22:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 0DF8E2F63A for ; Sat, 1 Sep 2018 15:22:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w81FMYAJ010010 for ; Sat, 1 Sep 2018 15:22:34 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w81FMYp0010009 for virtualization@FreeBSD.org; Sat, 1 Sep 2018 15:22:34 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230402] With buildworld, the system can not use swap Date: Sat, 01 Sep 2018 15:22:34 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Sep 2018 15:22:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230402 --- Comment #11 from Mark Millard --- (In reply to chris from comment #10) You are not explicit about what revision you are building. My experience is with head (12), not 11.x . If the following is supported: sysctl vm.pageout_oom_seq=3D120 then do that before starting the first build after booting. The default value of 12 is unlikely to work. Depending on what all is going on in your I/O environment, this may prove insufficient but it likely would get more of the build done. If the build does not complete, then investigating your I/O latencies becomes relevant. The figure is tied to how long FreeBSD tolerates low free RAM conditions. (This wording is a simplification.) FreeBSD does not swap running processes to gain more free RAM, only processes that are idle for a while. Another point is the use of -j4 or other such vs. -j1 . -j1 or other smaller figures are more likely to complete (use less memory and have fewer long-running processes at once). You were not explicit about your usage for this. As for the swap space sizing (1 page =3D 4*1024 Bytes): 1048576 pages is 1048576 * (4*1024) Bytes, so 4 GiBytes, not the 1 GiByte referenced. 924056 pages is 924056 * (4*1024) Bytes, so a little over 3.5 GiBytes. (Note the figures in the messages are system specific and can even change some from build revision to revision for the same system.) I'd recommend staying at or under the 3.5 GiByte figure. But going anywhere near 1 GiByte of swap is insufficient with 1 GiByte of RAM. 2 GiByte of swap should work with some room to spare. Is the reference to 512M of swap in another context similarly off by a factor of 4? If yes: 2 GiBytes of swap were in use. Otherwise? Again -j4 or other such vs. -j1 matters to the RAM+SWAP use and the number of long-running processes at once. I recommend using swap partitions and avoiding the use of swap files. (I've no clue which you are using.) --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Sep 1 15:50:53 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E697CFF335D for ; Sat, 1 Sep 2018 15:50:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 89D2F71455 for ; Sat, 1 Sep 2018 15:50:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 485E8FF335A; Sat, 1 Sep 2018 15:50:52 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37106FF3359 for ; Sat, 1 Sep 2018 15:50:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D37F57144F for ; Sat, 1 Sep 2018 15:50:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 380AB2F902 for ; Sat, 1 Sep 2018 15:50:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w81FophP059843 for ; Sat, 1 Sep 2018 15:50:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w81FopPr059842 for virtualization@FreeBSD.org; Sat, 1 Sep 2018 15:50:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230402] With buildworld, the system can not use swap Date: Sat, 01 Sep 2018 15:50:51 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Sep 2018 15:50:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230402 --- Comment #12 from Mark Millard --- (In reply to Mark Millard from comment #11) I forgot to mention limiting the linker (lld) to single threaded operation as a potential help relative to RAM usage during builds: LDFLAGS.lld+=3D -Wl,--no-threads in a make.conf or src.conf like file used for the likes of buildworld buildkernel activity. --=20 You are receiving this mail because: You are on the CC list for the bug.=