From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 8 20:13:16 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0BE1A106564A for ; Mon, 8 Dec 2008 20:13:16 +0000 (UTC) (envelope-from gofd-freebsd-virtualization@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id B1E0F8FC16 for ; Mon, 8 Dec 2008 20:13:15 +0000 (UTC) (envelope-from gofd-freebsd-virtualization@m.gmane.org) Received: from root by ciao.gmane.org with local (Exim 4.43) id 1L9mGx-0005nx-3N for freebsd-virtualization@freebsd.org; Mon, 08 Dec 2008 20:00:03 +0000 Received: from 93-138-120-23.adsl.net.t-com.hr ([93.138.120.23]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 08 Dec 2008 20:00:03 +0000 Received: from ivoras by 93-138-120-23.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 08 Dec 2008 20:00:03 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-virtualization@freebsd.org From: Ivan Voras Date: Mon, 08 Dec 2008 20:53:30 +0100 Lines: 34 Message-ID: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig29D131333AAFA203A42851E6" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-120-23.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.18 (Windows/20081105) X-Enigmail-Version: 0.95.7 Sender: news Subject: Exposing VM detection as sysctl X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2008 20:13:16 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig29D131333AAFA203A42851E6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Moving discussion from src-committers (<9bbcef730812081122r2b1fb938s6201aeee7756b884@mail.gmail.com>): Any objections to the patch: http://people.freebsd.org/~ivoras/diffs/subr_param.c.diff.txt ? (Of course, for symmetry, if we ever support any hosting for full virtualization or even for our monsters like vimage, another sysctl might be introduced as a bitmap that describes VM hosting abilities). --------------enig29D131333AAFA203A42851E6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkk9e0AACgkQldnAQVacBchcGACgiRzNFbaAQ3Kc8IGqQZ7zdgLl qN0An2knHyPFuQUo8dZQtVyyGJTR2lpJ =ZWzX -----END PGP SIGNATURE----- --------------enig29D131333AAFA203A42851E6-- From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 9 14:07:42 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E00FD1065678 for ; Tue, 9 Dec 2008 14:07:42 +0000 (UTC) (envelope-from duerrm@gmail.com) Received: from gv-out-0910.google.com (gv-out-0910.google.com [216.239.58.185]) by mx1.freebsd.org (Postfix) with ESMTP id 71C0A8FC13 for ; Tue, 9 Dec 2008 14:07:42 +0000 (UTC) (envelope-from duerrm@gmail.com) Received: by gv-out-0910.google.com with SMTP id n8so340865gve.39 for ; Tue, 09 Dec 2008 06:07:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type; bh=wrCR0ME6XEKDqiNcWwtfMm4jsC3jYU9B6rnMw3CsxZI=; b=fps+3BRCga0qXgF7mbsDpi6P7CCIKsjH2+3tX/+wissqKuHMFlDjDd8ZrvBV6LnEF0 TzyH755QcymyfCGQgTy69VvFfED7ukxJv5IsOLj0uNQbQbx9Lzov6IKHvDODUbkQZfE7 pgFM69f0Bp2m/pAbPKmoNx5WCES45vCO8EfVo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=F5ohYVoOSew6lUOiHwN0ltBXu+bhbNe7C4cksXTO85Xi4RzxKkl4JmGkSkxu0j1d57 GSonwtdi5XpeZA8nXz0HwAAzLu8NGldUHa99yaBHS3AHdkVLEtCYEQSafEP/tAPngxp3 07uZ231TYHWKLxlmTQxC/4HkJPvKJ6Vlvgfuk= Received: by 10.86.68.2 with SMTP id q2mr33794fga.3.1228830420148; Tue, 09 Dec 2008 05:47:00 -0800 (PST) Received: by 10.86.57.14 with HTTP; Tue, 9 Dec 2008 05:47:00 -0800 (PST) Message-ID: Date: Tue, 9 Dec 2008 14:47:00 +0100 From: "=?ISO-8859-1?Q?michael_d=FCrr?=" To: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Howto setup virtualisation X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Dec 2008 14:07:43 -0000 Hallo list, I recently discovered the IMUNES toolkit and found me more than fascinated by its functionallity. Now I'd love to deploy the most recent version of vimage. I already came accross Marko's website (http://imunes.tel.fer.hr/virtnet/). As I don't want to use the VMware images, but install everything native, I set up a FreeBSD 7.0 system and tried to compile against the provided snapshots. As you can imagine, my build failed :-( . Although I'm not that familiar with FreeBSD, I tried hard to compile the snapshots (as well as the provided patch) against different tags... Without success. My question: Can you tell me a release/tag which I can download from the src tree which comliles and for which I can use the GUI tools? Or is there any other simple solution to get all this running? Thank you! Michael From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 9 16:06:19 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78918106564A for ; Tue, 9 Dec 2008 16:06:19 +0000 (UTC) (envelope-from zec@icir.org) Received: from labs4.cc.fer.hr (labs4.cc.fer.hr [161.53.72.24]) by mx1.freebsd.org (Postfix) with ESMTP id DC5F18FC1D for ; Tue, 9 Dec 2008 16:06:18 +0000 (UTC) (envelope-from zec@icir.org) Received: from sluga.fer.hr (sluga.cc.fer.hr [161.53.72.14]) by labs4.cc.fer.hr (8.14.2/8.14.2) with ESMTP id mB9Fjga2000077 for ; Tue, 9 Dec 2008 16:45:44 +0100 (CET) Received: from [192.168.200.110] ([161.53.19.79]) by sluga.fer.hr with Microsoft SMTPSVC(6.0.3790.3959); Tue, 9 Dec 2008 16:45:09 +0100 From: Marko Zec To: freebsd-virtualization@freebsd.org Date: Tue, 9 Dec 2008 16:44:53 +0100 User-Agent: KMail/1.9.7 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200812091644.53757.zec@icir.org> X-OriginalArrivalTime: 09 Dec 2008 15:45:09.0847 (UTC) FILETIME=[1DDAEE70:01C95A15] X-Scanned-By: MIMEDefang 2.64 on 161.53.72.24 Cc: Subject: Re: Howto setup virtualisation X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Dec 2008 16:06:19 -0000 On Tuesday 09 December 2008 14:47:00 michael d=FCrr wrote: > Hallo list, > > I recently discovered the IMUNES toolkit and found me more than > fascinated by its functionallity. > > Now I'd love to deploy the most recent version of vimage. I already > came accross Marko's website (http://imunes.tel.fer.hr/virtnet/). As > I don't want to use the VMware images, but install everything native, > I set up a FreeBSD 7.0 system and tried to compile against the > provided snapshots. > > As you can imagine, my build failed :-( . Although I'm not that > familiar with FreeBSD, I tried hard to compile the snapshots (as well > as the provided patch) against different tags... Without success. > > My question: > > Can you tell me a release/tag which I can download from the src tree > which comliles and for which I can use the GUI tools? Or is there any > other simple solution to get all this running? Hi, builing and running http://imunes.net/virtnet/vimage_7-20081015.tgz on a 7.0-RELEASE or 7.1-PRERELEASE should be pretty straightforward:=20 just unpack the vimage_7 src tree; cd src/sys/i386/conf; config VIMAGE;=20 cd ../compile/VIMAGE; make depend; make; make install and you're done. You'll also need to compile the vimage(8) utility -> cd=20 src/usr.sbin/vimage; make; make install -> done. You'll most probably=20 want to upgrade your userland from 7.0-RELEASE to RELENG_7 or=20 RELENG_7_1 in order for the utilities like sockstat and netstat to work=20 properly - the simpler alternative would be doing the installation from=20 a recent 7.x snapshot ISO... =46or the GUI, the latest version can be fetched from anoncvs: cvs -d :pserver:anonymous@cvs.imunes.net:/usr/local/src/cvsroot co=20 imunes cd imunes; ./install.sh -> done Note that you need to have tk-8.4 installed for the GUI to work, and=20 most probably you'll want quagga and or xorp as well (from ports). Good luck, Marko From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 9 20:17:33 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16C691065675; Tue, 9 Dec 2008 20:17:33 +0000 (UTC) (envelope-from prvs=julian=2224d921f@elischer.org) Received: from smtp-outbound.ironport.com (smtp-outbound.ironport.com [63.251.108.112]) by mx1.freebsd.org (Postfix) with ESMTP id EFDD68FC12; Tue, 9 Dec 2008 20:17:32 +0000 (UTC) (envelope-from prvs=julian=2224d921f@elischer.org) Received: from unknown (HELO julian-mac.elischer.org) ([10.251.60.95]) by smtp-outbound.ironport.com with ESMTP; 09 Dec 2008 11:48:57 -0800 Message-ID: <493ECBAA.9030400@elischer.org> Date: Tue, 09 Dec 2008 11:48:58 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.18 (Macintosh/20081105) MIME-Version: 1.0 To: Marko Zec References: <200812080223.mB82N6iZ019068@repoman.freebsd.org> In-Reply-To: <200812080223.mB82N6iZ019068@repoman.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Kris Kennaway , Robert Watson , FreeBSD virtualization mailing list Subject: Re: PERFORCE change 154309 for review (etc.) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Dec 2008 20:17:33 -0000 Marko, I'm following along using p4 and the diff script. It looks good.. If it all seems to be working we should commit what you have and then we should get Kris and Robert to try some of their benchmarks.. to confirm the numbers you saw earlier. Kris, the change that is upcoming for the next vimage commit gathers the various networking globals in the system and aggregates them together into several functional structures. e.g. one for inet related vars and another for interface and basic networking infrastructure related vars. The thing we would like "independent" corroboration of is that the change to having these gathered together, and initialized by an explicit initialization routing does not give any significant performance difference for a representative set of networking tests. Kris and Robert, what tests would you suggest and is there a chance that we can get some "independent" testing done on this? From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 10 00:13:48 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E22A7106564A; Wed, 10 Dec 2008 00:13:48 +0000 (UTC) (envelope-from zec@freebsd.org) Received: from labs4.cc.fer.hr (labs4.cc.fer.hr [161.53.72.24]) by mx1.freebsd.org (Postfix) with ESMTP id 52D688FC08; Wed, 10 Dec 2008 00:13:47 +0000 (UTC) (envelope-from zec@freebsd.org) Received: from sluga.fer.hr (sluga.cc.fer.hr [161.53.72.14]) by labs4.cc.fer.hr (8.14.2/8.14.2) with ESMTP id mB9Noihd007395; Wed, 10 Dec 2008 00:50:44 +0100 (CET) Received: from [192.168.200.110] ([161.53.19.79]) by sluga.fer.hr with Microsoft SMTPSVC(6.0.3790.3959); Wed, 10 Dec 2008 00:50:11 +0100 From: Marko Zec To: Julian Elischer Date: Wed, 10 Dec 2008 00:49:57 +0100 User-Agent: KMail/1.9.7 References: <200812080223.mB82N6iZ019068@repoman.freebsd.org> <493ECBAA.9030400@elischer.org> In-Reply-To: <493ECBAA.9030400@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200812100049.57456.zec@freebsd.org> X-OriginalArrivalTime: 09 Dec 2008 23:50:11.0571 (UTC) FILETIME=[DFD59C30:01C95A58] X-Scanned-By: MIMEDefang 2.64 on 161.53.72.24 Cc: Robert Watson , FreeBSD virtualization mailing list Subject: Re: PERFORCE change 154309 for review (etc.) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Dec 2008 00:13:49 -0000 On Tuesday 09 December 2008 20:48:58 Julian Elischer wrote: > Marko, > > I'm following along using p4 and the diff script. It looks good.. The code in vimage-commit2 passed through a make universe run, though a new panic just emerged in head related to kmacy's changes to routing tbale locking. I feel somewhat reluctant to dump commit2 to svn while there's an unrelated but known panic that doesn't allow the system to boot, and furthermore bz and I wanted to double-check that all sysctls would still work exactly the same way before and after the change - a few days ago I noticed something strange but want to check it with the most recent head... > If it all seems to be working we should commit what you have > and then we should get Kris and Robert to try some of their > benchmarks.. to confirm the numbers you saw earlier. Yes it would be extremely usefull to have people doing unbiased tests of all kinds, though I think that the right time to commence testing would be in a week or two from now when we'll have all three different variable layout and referencing models in place in head, selectable as compile time option, as outlined at the bottom of bz's wiki page: http://wiki.freebsd.org/Image/KernelOptions Marko > Kris, the change that is upcoming for the next vimage commit > gathers the various networking globals in the system and > aggregates them together into several functional structures. > > e.g. one for inet related vars and another for interface and basic > networking infrastructure related vars. > > The thing we would like "independent" corroboration of is that > the change to having these gathered together, and initialized by > an explicit initialization routing does not give any significant > performance difference for a representative set of networking > tests. > > Kris and Robert, what tests would you suggest and is there a chance > that we can get some "independent" testing done on this? From owner-freebsd-virtualization@FreeBSD.ORG Thu Dec 11 22:32:54 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C78A1065676; Thu, 11 Dec 2008 22:32:54 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 0497A8FC18; Thu, 11 Dec 2008 22:32:53 +0000 (UTC) (envelope-from des@des.no) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id D1D2E6D43F; Thu, 11 Dec 2008 22:32:52 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id ADCA7844C0; Thu, 11 Dec 2008 23:32:52 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Philipp Wuensche References: <20081201085229.D80401@maildrop.int.zabbadoz.net> <20081201122937.81475f0zhfsjya4o@webmail.leidinger.net> <6ae50c2d0812021800x791d2cfeh45d590de120f76df@mail.gmail.com> <1228483574.2805.499.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> <86skp2l804.fsf@ds4.des.no> <1228507529.2805.539.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> <49418BD9.8080105@h3q.com> Date: Thu, 11 Dec 2008 23:32:52 +0100 In-Reply-To: <49418BD9.8080105@h3q.com> (Philipp Wuensche's message of "Thu, 11 Dec 2008 22:53:29 +0100") Message-ID: <867i66s5pn.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD virtualization mailing list , alexus , freebsd-current@freebsd.org, Alexander Leidinger , "Bjoern A. Zeeb" , freebsd-jail@freebsd.org, bseklecki@collaborativefusion.com Subject: Re: HEADS UP: r185435 multi-IPv4/v6/no-IP jails in HEAD X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Dec 2008 22:32:54 -0000 Philipp Wuensche writes: > Not entirely true, the jls output is totaly different than before and > breaks third-party applications like jailaudit and ezjail. > > It is uneasy to parse too. jls | tail +3 | while read line ; do set $line if [ $# =3D 3 ] ; then echo "jail $1 (name $2 root $3) IPs:" elif [ $# =3D 1 ] ; then echo " $1" else echo "huh?" fi done DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-virtualization@FreeBSD.ORG Thu Dec 11 22:20:12 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D4AA1065670 for ; Thu, 11 Dec 2008 22:20:12 +0000 (UTC) (envelope-from cryx-freebsd@h3q.com) Received: from mail.h3q.com (mail.h3q.com [213.73.89.199]) by mx1.freebsd.org (Postfix) with ESMTP id 7B6CD8FC08 for ; Thu, 11 Dec 2008 22:20:11 +0000 (UTC) (envelope-from cryx-freebsd@h3q.com) Received: (qmail 86756 invoked from network); 11 Dec 2008 21:53:29 -0000 Received: from unknown (HELO goa.local) (smtpsend@85.179.28.10) by mail.h3q.com with AES256-SHA encrypted SMTP; 11 Dec 2008 21:53:29 -0000 Message-ID: <49418BD9.8080105@h3q.com> Date: Thu, 11 Dec 2008 22:53:29 +0100 From: Philipp Wuensche User-Agent: Thunderbird 2.0.0.18 (Macintosh/20081105) MIME-Version: 1.0 To: bseklecki@collaborativefusion.com References: <20081201085229.D80401@maildrop.int.zabbadoz.net> <20081201122937.81475f0zhfsjya4o@webmail.leidinger.net> <6ae50c2d0812021800x791d2cfeh45d590de120f76df@mail.gmail.com> <1228483574.2805.499.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> <86skp2l804.fsf@ds4.des.no> <1228507529.2805.539.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> In-Reply-To: <1228507529.2805.539.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Thu, 11 Dec 2008 22:49:48 +0000 Cc: FreeBSD virtualization mailing list , Alexander Leidinger , alexus , freebsd-current@freebsd.org, "Bjoern A. Zeeb" , freebsd-jail@freebsd.org Subject: Re: HEADS UP: r185435 multi-IPv4/v6/no-IP jails in HEAD X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Dec 2008 22:20:12 -0000 Brian A. Seklecki wrote: > On Fri, 2008-12-05 at 20:47 +0100, Dag-Erling Smørgrav wrote: >> The question is, does it change existing behavior, or just add new >> functionality? > > The syntax semantics should be backward compatible, so likely the > latter. Not entirely true, the jls output is totaly different than before and breaks third-party applications like jailaudit and ezjail. It is uneasy to parse too. greetings, Philipp From owner-freebsd-virtualization@FreeBSD.ORG Fri Dec 12 03:23:00 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A0471065670 for ; Fri, 12 Dec 2008 03:23:00 +0000 (UTC) (envelope-from antipsychic@gmail.com) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.234]) by mx1.freebsd.org (Postfix) with ESMTP id EF59D8FC1F for ; Fri, 12 Dec 2008 03:22:59 +0000 (UTC) (envelope-from antipsychic@gmail.com) Received: by rv-out-0506.google.com with SMTP id b25so1173417rvf.43 for ; Thu, 11 Dec 2008 19:22:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:mime-version:x-priority :content-type:message-id:content-transfer-encoding:from:subject:date :to:x-mailer; bh=upGjaBvPx5F9eLQWj/5c85yRPW3oOLKxbYNahvhC38c=; b=JoCTjjuUyDStErlDuIZiIXLV5rcSL0yoXBlfKe+DT6Rf0wUTxdHQmz6xYm29dv5Aed As/skcaQjNAqmDccF0DYvaaufCsgFWHgEsj3Qr8+t5SHyjVcvZfMHaRKdSombctZwpNj da8grKni+UdNByTxkEDr9XFYs+6P6xMW6HXDs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:x-priority:content-type:message-id :content-transfer-encoding:from:subject:date:to:x-mailer; b=HGCIhza2uQAWYAowSZs3QAwl3Zp6YnV/uUu/pLp7iQnHia4t9zuTzCjWOIVOC+/ZFO yxAvL1nRlHqrOKlc63l93LtAR4G2zkZQdCPDlLEDceiUiUGlttj4sg/j6O1ww8f7JBNZ JFotv16t3fQsWyrWcdJ2WU0mZO12DwCVb3HJc= Received: by 10.142.157.9 with SMTP id f9mr1118121wfe.87.1229050323702; Thu, 11 Dec 2008 18:52:03 -0800 (PST) Received: from ?10.10.16.37? (pool.cloudshield.com [63.150.56.65]) by mx.google.com with ESMTPS id 30sm4697887wfd.44.2008.12.11.18.52.02 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 11 Dec 2008 18:52:03 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v753.1) X-Priority: 3 Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jake Thalacker Date: Thu, 11 Dec 2008 18:52:00 -0800 To: freebsd-virtualization@freebsd.org X-Mailer: Apple Mail (2.753.1) Subject: SMP refcount_release panic X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2008 03:23:00 -0000 Sorry, this is overdue on being reported. I've encountered a panic with VIMAGE from a patched build of current from around mid September. I tracked the problem to the second call to refcount_release in kern_vimage.c:vi_td_ioctl(). Just for clarity I determined this by making 3 refcount_release functions to replace the calls including the one in kern_prot.c:crfree(). I noted that throughout the kernel code prior to all kern_prot.c:crcopy() calls there is a PROC_LOCK and proceeded by a PROC_UNLOCK afterward. I found that there was one occurrence where this was missing in netncp/ ncp_comm.c:ncp_conn_alloc(). I added it and that did not resolve the panic but figured it's still probably needed. It's probably poor form but I ended up commenting out the second call and have had no panics since, had to move on at that point. The panic is very reproducible with the following script on an SMP system. I have not observed any panics on non-SMP systems. _vimage="rc-panic-$$" vimage -c ${_vimage} _cnt=0 _fw=1 _loopcount=64 while [ ${_cnt} -lt ${_loopcount} ]; do vimage ${_vimage} sysctl net.inet.ip.forwarding=${_fw} if [ ${_fw} -eq 1 ]; then _fw=0 else _fw=1 fi _cnt=$(expr ${_cnt} '+' 1) done If there's any further information I can provide just let me know, or any testing of new code I'd gladly help out. Thanks for all your efforts! Jake From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 13 19:54:12 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ABB4D1065670; Sat, 13 Dec 2008 19:54:12 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mail.cksoft.de (mail.cksoft.de [62.111.66.27]) by mx1.freebsd.org (Postfix) with ESMTP id 37C9A8FC08; Sat, 13 Dec 2008 19:54:12 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from localhost (amavis.str.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id CBAC441C64A; Sat, 13 Dec 2008 20:35:05 +0100 (CET) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([62.111.66.27]) by localhost (amavis.str.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id LuSxzKuBSDNS; Sat, 13 Dec 2008 20:35:05 +0100 (CET) Received: by mail.cksoft.de (Postfix, from userid 66) id 6AADA41C615; Sat, 13 Dec 2008 20:35:05 +0100 (CET) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id 555864448D5; Sat, 13 Dec 2008 19:33:54 +0000 (UTC) Date: Sat, 13 Dec 2008 19:33:53 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, FreeBSD virtualization mailing list In-Reply-To: <200812131913.mBDJD38C037353@svn.freebsd.org> Message-ID: <20081213191345.M97918@maildrop.int.zabbadoz.net> References: <200812131913.mBDJD38C037353@svn.freebsd.org> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org, FreeBSD current mailing list Subject: HEADS UP: vimage - virtualized global variables in the network stack X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: FreeBSD virtualization mailing list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Dec 2008 19:54:12 -0000 On Sat, 13 Dec 2008, Bjoern A. Zeeb wrote: Hi, > Author: bz > Date: Sat Dec 13 19:13:03 2008 > New Revision: 186048 > URL: http://svn.freebsd.org/changeset/base/186048 > > Log: > Second round of putting global variables, which were virtualized > but formerly missed under VIMAGE_GLOBAL. > > Put the extern declarations of the virtualized globals > under VIMAGE_GLOBAL as the globals themsevles are already. > This will help by the time when we are going to remove the globals > entirely. As some of you might have noticed that Marko's last commit for the first time in the series of vimage commits was an actual functional change. By default HEAD is no longer using the globals. With my commit the current set of virtualized variables is assumed to be "clean". This basically means three things: 1) The former globals and their externs are all under #ifdef VIMAGE_GLOBALS 2) The same variables are present in a 'container struct' 3) The initialization of those is done from 'constructor ("init") functions' This state of having the variables in parallel, global and in the container struct, will be maintained for another (short) time until the entire virtualization framework is in. This is needed, so that all three possible states can be benchmarked from exactly the same code changeset. For developers comitting new code or changing code it is important to properly add virtualized variables in the way that: 1) the globals and externs (if needed) are added/kept in sync as both a) globals under #ifdef VIMAGE_GLOBALS and b) to the appropriate container struct + the V_ macro. When used somewhere in code one has to use the V_foobarbaz version. 2) Any new virtualized globals must not be directly initialized. They have to be initialized from a contructor function (which is usually there already). If you are confused about some of the terms etc. follow the links in the "Some documentation" section on the wiki: http://wiki.freebsd.org/Image The "Vimage Coding - beginners guide / FAQ" tries to answer the 101 questions. For the beaf you'll find the link to a document in perforce with the last question (that you may already know). We'll try to enhance this as we get questions or the integration goes on. In case of questions or suggestions ideally follow-up on freebsd-virtualization@ . /bz -- Bjoern A. Zeeb The greatest risk is not taking one. From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 13 19:57:54 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 19E411065737 for ; Sat, 13 Dec 2008 19:57:54 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.187]) by mx1.freebsd.org (Postfix) with ESMTP id 9CE188FC14 for ; Sat, 13 Dec 2008 19:57:53 +0000 (UTC) (envelope-from max@love2party.net) Received: from vampire.homelinux.org (dslb-088-066-024-089.pools.arcor-ip.net [88.66.24.89]) by mrelayeu.kundenserver.de (node=mrelayeu8) with ESMTP (Nemesis) id 0ML31I-1LBaQP2hyG-0002O9; Sat, 13 Dec 2008 20:45:17 +0100 Received: (qmail 31355 invoked from network); 13 Dec 2008 19:45:17 -0000 Received: from fbsd8.laiers.local (192.168.4.151) by ns1.laiers.local with SMTP; 13 Dec 2008 19:45:17 -0000 From: Max Laier Organization: FreeBSD To: FreeBSD virtualization mailing list Date: Sat, 13 Dec 2008 20:45:16 +0100 User-Agent: KMail/1.10.1 (FreeBSD/8.0-CURRENT; KDE/4.1.1; i386; ; ) References: <200812131913.mBDJD38C037353@svn.freebsd.org> <20081213191345.M97918@maildrop.int.zabbadoz.net> In-Reply-To: <20081213191345.M97918@maildrop.int.zabbadoz.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200812132045.17207.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1/Ij2V3e3xO0mfz31az75+nR9M+cw1orHT/ZRm 5Vx8xDsSsZJzZAjtuYFL+V+aQMI1KYrXo2bK37Eagx1H3eIF7r 7yVbWzWON45I1jxy/3ljw== X-Mailman-Approved-At: Sat, 13 Dec 2008 20:08:06 +0000 Cc: svn-src-head@freebsd.org, freebsd-net@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, FreeBSD current mailing list Subject: Re: HEADS UP: vimage - virtualized global variables in the network stack X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Dec 2008 19:57:54 -0000 On Saturday 13 December 2008 20:33:53 Bjoern A. Zeeb wrote: ... > This state of having the variables in parallel, global and in the > container struct, will be maintained for another (short) time until > the entire virtualization framework is in. This is needed, so that > all three possible states can be benchmarked from exactly the same > code changeset. > > > For developers comitting new code or changing code it is important to > properly add virtualized variables in the way that: > 1) the globals and externs (if needed) are added/kept in sync as both > a) globals under #ifdef VIMAGE_GLOBALS and b) to the appropriate > container struct + the V_ macro. > When used somewhere in code one has to use the V_foobarbaz version. Is there (an easy) way to have the tinderbox build every other run without VIMAGE_GLOBALS so that the most obvious error (global available, but not in the container struct - or the other way around) can be warned about? -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 13 20:10:07 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7744410656A3; Sat, 13 Dec 2008 20:10:07 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mail.cksoft.de (mail.cksoft.de [62.111.66.27]) by mx1.freebsd.org (Postfix) with ESMTP id F28588FC17; Sat, 13 Dec 2008 20:10:06 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from localhost (amavis.str.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id 2048F41C6A3; Sat, 13 Dec 2008 21:10:06 +0100 (CET) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([62.111.66.27]) by localhost (amavis.str.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id AL8qj7J-9cEK; Sat, 13 Dec 2008 21:10:05 +0100 (CET) Received: by mail.cksoft.de (Postfix, from userid 66) id BC8A041C6A1; Sat, 13 Dec 2008 21:10:05 +0100 (CET) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id 1A8434448D5; Sat, 13 Dec 2008 20:07:38 +0000 (UTC) Date: Sat, 13 Dec 2008 20:07:38 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Max Laier In-Reply-To: <200812132045.17207.max@love2party.net> Message-ID: <20081213195343.V97918@maildrop.int.zabbadoz.net> References: <200812131913.mBDJD38C037353@svn.freebsd.org> <20081213191345.M97918@maildrop.int.zabbadoz.net> <200812132045.17207.max@love2party.net> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: src-committers@freebsd.org, FreeBSD current mailing list , FreeBSD virtualization mailing list , freebsd-net@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: HEADS UP: vimage - virtualized global variables in the network stack X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Dec 2008 20:10:07 -0000 On Sat, 13 Dec 2008, Max Laier wrote: > On Saturday 13 December 2008 20:33:53 Bjoern A. Zeeb wrote: > ... >> This state of having the variables in parallel, global and in the >> container struct, will be maintained for another (short) time until >> the entire virtualization framework is in. This is needed, so that >> all three possible states can be benchmarked from exactly the same >> code changeset. >> >> >> For developers comitting new code or changing code it is important to >> properly add virtualized variables in the way that: >> 1) the globals and externs (if needed) are added/kept in sync as both >> a) globals under #ifdef VIMAGE_GLOBALS and b) to the appropriate >> container struct + the V_ macro. >> When used somewhere in code one has to use the V_foobarbaz version. > > Is there (an easy) way to have the tinderbox build every other run without > VIMAGE_GLOBALS so that the most obvious error (global available, but not in > the container struct - or the other way around) can be warned about? Without VIMAGE_GLOBALS is the default; we have been building this for a few days already. The flip had been so smoothly that almost noone had really noticed. Marko has done a really great job! Thus my HEADS UP now after I am confident enough that (almost) all places were caught and clean. In case you want to check yourself you can simply put a file into one or multiple archs conf dir that looks like: ------------------------------------------------------------------------ > cat sys/amd64/conf/LINT-VIMAGE_GLOBALS include LINT ident LINT-VIMAGE_GLOBALS options VIMAGE_GLOBALS ------------------------------------------------------------------------ I am doing that build every other day from now to catch the possible error of a virtualized variable in the container struct w/o the global. But as this is the least problematic case I do not want to commit the kernel configuration as it'll make builds longer for everyone, etc. The more problematic cases that builds cannot catch are: - static initialization of a virtualized 'global'. - a newly added virtualized 'global' that is not under #ifdef VIMAGE_GLOBALS I have scripts to identify the latter already. The former will only be caught be either code inspection or "unexpected results" when running. /bz -- Bjoern A. Zeeb The greatest risk is not taking one. From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 13 20:27:17 2008 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C195106570C; Sat, 13 Dec 2008 20:27:17 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 259428FC13; Sat, 13 Dec 2008 20:27:17 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.2/8.14.1) with ESMTP id mBDKOPGs092903; Sat, 13 Dec 2008 13:24:25 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Sat, 13 Dec 2008 13:24:25 -0700 (MST) Message-Id: <20081213.132425.41724046.imp@bsdimp.com> To: max@love2party.net From: Warner Losh In-Reply-To: <200812132045.17207.max@love2party.net> References: <200812131913.mBDJD38C037353@svn.freebsd.org> <20081213191345.M97918@maildrop.int.zabbadoz.net> <200812132045.17207.max@love2party.net> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 13 Dec 2008 20:31:29 +0000 Cc: src-committers@FreeBSD.org, current@FreeBSD.org, freebsd-virtualization@FreeBSD.org, freebsd-net@FreeBSD.org, svn-src-all@FreeBSD.org, svn-src-head@FreeBSD.org Subject: Re: HEADS UP: vimage - virtualized global variables in the network stack X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Dec 2008 20:27:17 -0000 From: Max Laier Subject: Re: HEADS UP: vimage - virtualized global variables in the network stack Date: Sat, 13 Dec 2008 20:45:16 +0100 > On Saturday 13 December 2008 20:33:53 Bjoern A. Zeeb wrote: > ... > > This state of having the variables in parallel, global and in the > > container struct, will be maintained for another (short) time until > > the entire virtualization framework is in. This is needed, so that > > all three possible states can be benchmarked from exactly the same > > code changeset. > > > > > > For developers comitting new code or changing code it is important to > > properly add virtualized variables in the way that: > > 1) the globals and externs (if needed) are added/kept in sync as both > > a) globals under #ifdef VIMAGE_GLOBALS and b) to the appropriate > > container struct + the V_ macro. > > When used somewhere in code one has to use the V_foobarbaz version. > > Is there (an easy) way to have the tinderbox build every other run without > VIMAGE_GLOBALS so that the most obvious error (global available, but not in > the container struct - or the other way around) can be warned about? This actually points out why the 'tinderbox' name is bogus for the universe plus failure: universe builds all the kernels. Tinderbox builds LINT only. Warner