From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 23 19:43:52 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54288E43; Sun, 23 Nov 2014 19:43:52 +0000 (UTC) Received: from mail-qg0-x235.google.com (mail-qg0-x235.google.com [IPv6:2607:f8b0:400d:c04::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04E62D1D; Sun, 23 Nov 2014 19:43:52 +0000 (UTC) Received: by mail-qg0-f53.google.com with SMTP id q108so5818244qgd.26 for ; Sun, 23 Nov 2014 11:43:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=Nz3B8zR0Ql/Jfp4PP9oeeLauC0+y04rS+G5id7IQKBA=; b=baWeg9aJkegUVcb9XyMzNmjwTxoDX78xiDy6wjw180NNwlXdn+iLQT6q8H5Uzvs5mk PQromvoYbrm3YYATw6EfYKy/gZJ1nkL8KA8YLM+8r3BmHRbDxFgZ7CWC1Jf+2fCvwDpO G/7C0Pxq4/nWc/gw1HLVT1x6EKb/m1g82Y3SkiLz+jXmfgRfsEjYDhGIU8DaI9B8gvbK c4SZKhGi7+kSaxpGOnvaETD63hOcOapOvntglL4Wmiw4LHK7HeTwr2Gvix0iLF3Y4iVh 4aXMF1yzHwhBp99W735hniJAJsPkewc+Ofvr5fQ17Tm6hhDlWfICKnDXv/yI5euKkdm1 tylw== X-Received: by 10.140.96.203 with SMTP id k69mr23537525qge.33.1416771831117; Sun, 23 Nov 2014 11:43:51 -0800 (PST) Received: from hbsd-dev-local ([73.173.99.185]) by mx.google.com with ESMTPSA id p106sm10046349qgd.36.2014.11.23.11.43.50 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 23 Nov 2014 11:43:50 -0800 (PST) Date: Sun, 23 Nov 2014 14:43:49 -0500 From: Shawn Webb To: Peter Grehan Subject: Re: bhyve cannot allocate memory Message-Id: <20141123144349.84f648dcfc1169bf952c7ee7@gmail.com> In-Reply-To: <54715B13.9020303@freebsd.org> References: <20141122215245.d9380cc4e43cb5e60d479009@gmail.com> <20141122220202.09523b0ae828993174af05d8@gmail.com> <5471513C.6040400@freebsd.org> <54715438.3090905@freebsd.org> <54715822.2010309@freebsd.org> <54715911.1090100@freebsd.org> <54715B13.9020303@freebsd.org> X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2014 19:43:52 -0000 On Sat, 22 Nov 2014 19:57:07 -0800 Peter Grehan wrote: > Hi Shawn, > > > Interesting. I'll have to do more digging. Because removing map_at_zero > > support is the same as keeping it at the default of 0. It's not possible > > that our ASLR implementation is affecting bhyve, since our ASLR > > implementation is in sys_mmap and the elf image activator. At this > > stage, bhyve's vmm.ko is directly accessing vm_map_*, which we haven't > > touched. > > One thing you may be able to try is ktrace the bhyveload process and > see which syscall is failing. Oliver fixed the problem. bhyve's vmm.ko uses VM_MIN_ADDRESS, which defaults to NULL in FreeBSD. We modified that to be at 64k to prevent NULL dereferences. Combine that with the removal of map_at_zero, NULL deref is not possible anymore. But since we bumped VM_MIN_ADDRESS up, bhyve's vmm.ko no longer attempts to map at NULL (which it absolutely needs to). The change Oliver made simply uses 0 instead of VM_MIN_ADDRESS. If this is a change FreeBSD wants upstream, we'd love it merged upstream. Commit for reference: https://github.com/HardenedBSD/hardenedBSD/commit/23bd719ce1e3a8cc42fc8317b1c7c6d9e74dcba0 Thanks, Shawn