From owner-freebsd-virtualization@freebsd.org Sun Mar 20 03:09:39 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F3B2AD6421 for ; Sun, 20 Mar 2016 03:09:39 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: from quine.pinyon.org (quine.pinyon.org [65.101.5.249]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD72E1DCB for ; Sun, 20 Mar 2016 03:09:38 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: by quine.pinyon.org (Postfix, from userid 122) id DE757160230; Sat, 19 Mar 2016 20:09:30 -0700 (MST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on quine.pinyon.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 Received: from feyerabend.n1.pinyon.org (h5.esturion.net [65.101.5.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by quine.pinyon.org (Postfix) with ESMTPSA id B028B160160 for ; Sat, 19 Mar 2016 20:09:28 -0700 (MST) Subject: Re: UFS vs. ZFS inside bhyve hosted on ZFS To: freebsd-virtualization@freebsd.org References: <56ECAFEB.8060305@pinyon.org> From: "Russell L. Carter" Message-ID: <56EE1468.5050503@pinyon.org> Date: Sat, 19 Mar 2016 20:09:28 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.7.0 MIME-Version: 1.0 In-Reply-To: <56ECAFEB.8060305@pinyon.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Mar 2016 03:09:39 -0000 On 03/18/16 18:48, Russell L. Carter wrote: > Greetings, > > So I am becoming quite enamored of an 11-current bhyve guest installed > with ZFS root running on a 10-stable host with ZFS and driven by an > AMD FX-8320. However the 3 cpus I give it seem to translate to a lot > of overhead on the host when building the -current world, determined > by drinking beer and watching htop on both. The guest is running a > GENERIC-NODEBUG kernel, and I'm building with MALLOC_PRODUCTION=yes. > The host is much leaner. > > So I am wondering if UFS in the -current guest might be better > overall. I can certainly do a multiple hour experiment, installing > a new guest with UFS root, but since I am new to this, perhaps there is > conventional wisdom about ZFS vs. UFS in the guest? Maybe UFS in the > guest requires less cpu resources from the host? Or not? I had an off-list suggestion to go ahead and DO THE WORK, so I'm now fairly far down the path. I have two bhyve VMs configured identically, both running r297047, each configured with 15G of RAM and 3 cpus, verified by examining 'sysctl hw' from within the running guest. One has the default ZFS install, the other UFS. However, # cd /usr/src && make buildworld -j after 15 minutes or so freezes the -current-UFS guest with the following console output: root@vm1:~ # ahcich0: Timeout on slot 29 port 0 ahcich0: is 00000008 cs 00000000 ss 00000000 rs bfffffff tfd 50 serr 00000000 cmd 0001dd17 (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 70 dc 3a 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: Command timeout (ada0:ahcich0:0:0:0): Retrying command I'm using the defaults for the iohyve script, which are easily extracted if required. The -current-ZFS guest has been completely solid, building ports and world/kernel across multiple revisions. That said I am completely new to this so likely have misconfigured something. Thanks, Russell