From owner-freebsd-virtualization@FreeBSD.ORG Thu Mar 20 21:40:38 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88378CD4; Thu, 20 Mar 2014 21:40:38 +0000 (UTC) Received: from freebsd.czest.pl (freebsd.czest.pl [212.87.224.105]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C72F48D8; Thu, 20 Mar 2014 21:40:37 +0000 (UTC) Received: from freebsd.czest.pl (freebsd.czest.pl [212.87.224.105]) by freebsd.czest.pl (8.14.5/8.14.5) with ESMTP id s2KLcmkd010431; Thu, 20 Mar 2014 21:38:49 GMT (envelope-from wkoszek@freebsd.czest.pl) Received: (from wkoszek@localhost) by freebsd.czest.pl (8.14.5/8.14.5/Submit) id s2KLcmD7010430; Thu, 20 Mar 2014 21:38:48 GMT (envelope-from wkoszek) Date: Thu, 20 Mar 2014 21:38:48 +0000 From: "Wojciech A. Koszek" To: Craig Rodrigues Subject: Re: Google Summer of Code 2014 question Message-ID: <20140320213848.GU37327@FreeBSD.org> References: <532A0DE6.2020801@pitt.edu> <201403201124.07472.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-0.4 required=5.0 tests=RP_MATCHES_RCVD, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on freebsd.czest.pl X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (freebsd.czest.pl [212.87.224.105]); Thu, 20 Mar 2014 21:38:55 +0000 (UTC) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 21:40:38 -0000 On Thu, Mar 20, 2014 at 01:54:53PM -0700, Craig Rodrigues wrote: > On Thu, Mar 20, 2014 at 8:24 AM, John Baldwin wrote: > > > > This isn't a bad place to ask this question. I think a person you might > > want to talk to is Craig Rodrigues (cc'd) who has recently setup Jenkins > > with the goal of doing automated testing using virtual machines. I'm not > > sure exactly what design he is considering, but he is the person to ask. > > Last year I signed up to be a GSoC mentor, but this year I unfortunately > do not have time, so haven't done so. Unfortunately, since I did not > sign up as a mentor, I can't view your proposal on the google-melange web site. > > If you look at the following link, you can see the March 13 > presentation which describes > some work I did setting up Jenkins and BHyve for continuous > integration and testing on FreeBSD: > > https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups > > If you look at this link, item 22: > > https://wiki.freebsd.org/Jenkins#Jenkins_for_FreeBSD_status > > you will see that we are trying to work at integrating Jenkins with the Kyua > framework, so that Jenkins can run Kyua, and report the results in the > Jenkins web UI. > > Jenkins is used by hundreds of companies, not just in the FreeBSD world. > Kyua is gaining popularity in the FreeBSD community, and is already used by > major companies (EMC/Isilon and Spectralogic are the ones that I know of). > > There are already many systems out there for doing automation, > starting/stopping VM's, > and running tests. I'm focusing on integrating as many existing > technologies and > getting them to run under FreeBSD to put together a more modern > continuous integration and testing > pipeline for the FreeBSD project. > > I cannot comment on the specifics your proposal without reading it. > However, I don't see a lot of value in coming up with a new Domain > Specific Language > for starting/stopping VM's and sending commands to VM's. There are > already many people > doing this already with other frameworks and automation API's. For > example, libvirt ( http://libvirt.org ) > is one popular API specifically for controlling VM's. > > In terms of frameworks that can send remote commands to VM's, there > are already plenty > of automation systems to choose from: Puppet, Chef, SaltStack, > CFEngine are some examples. > > Creating a new Domain Specific Language for VM control sounds like a > more academic/research exercise, > so if you can get someone to mentor you for such a project, maybe you > can target the results > of your work for publication, if that is of interest to you. John, Craig, When I initially proposed the idea: https://wiki.freebsd.org/SummerOfCode2014#TEST-o-steron_for_FreeBSD_.28node.js.2FLua.29 it was meant to be mainly a regression testing tool for FreeBSD. Something that would let me build and run continuous builds for FreeBSD ARM (ZEDBOARD kernel), and then make a SD image out of it, which I can boot in QEMU/real hardware. It would be nice if we payed attention to BHyVE in this project, but not limit ourselves to it. E.g.: ARM testing was my original drive towards proposing this. For such a system I'd like to be able to recognize what is being printed on UART at the boot time. Then register the output in a file and later compare it. And I'd like enough expressiveness to say: "if there's no login screen after 30s of bot time, fail and let me know what the screen content at the time of failure was". DSL isn't a must here, I agree, but you'll be limited in testing otherwise. Unless some technology already has what I want, I suggested getting this functionality: vm0 = vm.start("vm0"); vm1 = vm.start("vm1"); vm0.power(1); vm1.power(1); vm0.login(); vm1.login(); vm0.cmd("ifconfig ....."); vm1.cmd("ifconfig ...."); vm0.cmd("ping vm1"); vm0.register(function(data) { if (data =~ /*Not*/) { error(); } }); Once this would work for VMs, I'd be interesting in proposing the creation of the physical infrastructure for continuous FreeBSD testing. We could start this from Atom-based PC and Raspberry PI netbooting latest FreeBSD. If John sees it in a similar way and he'd like to explore this path, I'd be very interested in mentoring this idea. But once again: only if there isn't something of a similar functionality. Thanks, -- Wojciech A. Koszek wkoszek@FreeBSD.czest.pl http://FreeBSD.czest.pl/~wkoszek/