From owner-freebsd-stable@freebsd.org Fri Oct 14 13:08:56 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E50C5C11728; Fri, 14 Oct 2016 13:08:56 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-oi0-x241.google.com (mail-oi0-x241.google.com [IPv6:2607:f8b0:4003:c06::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8BE3391; Fri, 14 Oct 2016 13:08:56 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-oi0-x241.google.com with SMTP id d132so7502769oib.2; Fri, 14 Oct 2016 06:08:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=DzbAULD0FK8r0nJDWyxtYJQ0zBLG9Erw+7jm9OkOCPI=; b=pNb3r3tP4i14GiQbQkKCAkg1k//VU0L0+ub47TseVSAQ9K3Cct8Y+3CQYP2a7gzKH/ a6F029BJIqyAZYsSvJuUljsuznJT/TF5c4BdwL/OkvyeoNhEqPIkg/Rls8r+oYk9+5Q3 Hl+5kHqzvbAJDObtL0cyK1QYTvhNCHRAnHK60pL7Eta2ytbYi6G3xwQHo7zqVbeuz9Pu VygcnnACANnJvFKQRFHZn/Y/OO+swr9Wto94aDgOydeYZN7DDnaq79M9Pqhz/FTo+MPz CvKtW6sIoTVg7ji+Bm9nkoOZfa6OnBaL2NTuxi5PpwIFiB30iHxaTilu7C6eJsxBjsW6 PP4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=DzbAULD0FK8r0nJDWyxtYJQ0zBLG9Erw+7jm9OkOCPI=; b=lDuhyaIKhys0KT6AigvXF5CSewSLCi+0wv5S7WuE1J2FMLUmfds+ZuR7sSEsy9bwge 3ZLuFpudzzJAaRuF2hOmFenSUab6Vs0kg5+4R0KAr7GG3C6mk4tgZ9JSWv8tJ72qT8qC 8QtBsCQCh+NHnja3zMKxhtVrgSU65yssXDI23BLPR+StdR2BVhiGOp9Dv7Zqnd626w0O /suOzo9yy6+6uKVqHBDUGsqidsXGyNypfVIvdv04CEjobtEoRIbqiJBTZ3lmQdpwH17N kUqnhpQYSitaCBT7aocIS+dQIuWjLdCklfkLb/IPHM9Rs/pO25dgZ0ga+NHHSEJ5qDRs DxSg== X-Gm-Message-State: AA6/9RkIhejN+qx6sydDXtsmC6LgTl0WRJsvOxc7F3jV0PyUNaTv4D/VYq4fET1Sm1bHDi3bmF/LTu3o0n/wpQ== X-Received: by 10.157.42.167 with SMTP id e36mr5549928otb.232.1476450536015; Fri, 14 Oct 2016 06:08:56 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.61.52 with HTTP; Fri, 14 Oct 2016 06:08:55 -0700 (PDT) In-Reply-To: <58009EB4.30708@omnilan.de> References: <58009EB4.30708@omnilan.de> From: Vincenzo Maffione Date: Fri, 14 Oct 2016 15:08:55 +0200 Message-ID: Subject: Re: vale-ctl(-8), ifconfig(8), SIOCAIFADDR: Invalid argument [utilizing netmap(4) providing virtual switches+interfaces to BHyVe] To: Harry Schmalzbauer Cc: FreeBSD Net , FreeBSD Stable , Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 13:08:57 -0000 Hi, Thanks for your feedback. 2016-10-14 11:00 GMT+02:00 Harry Schmalzbauer : > Dear all, > > I found great papers about netmap(4)s desigen and implementation > details, and I'm sure it's one other masterpeace of rizzo-quality :-) > Thanks to all participants for that great code! > > To be honest, I haven't read all of that, because I'm short in time and > my first mission is to see if FreeBSD 11 will replace some of my ESXi > machines. > > One key element seems netmap(4). > It's quiet hard to find userland documentation. > > So far, I've discovered that there are three essential tools waiting in > _usr/src/tools/tools/netmap_ to be compiled > (resulting in *./vale-ctl*, *./bridge*, *./pkt-gen*) > > While the latter is often referenced in netmap(4) documentation, it's > not of interest for me, because I'll be doing real-world performance > tests and I'm convinced that all the impressive numbers presented in the > netmap documentation are valid :-) > > So *vale-ctl(-8)* seems to be of interest (I'm using (-8) becaus > currently there is no man8 part (I guess that's the reason for these > tools not beeing integrated into base binaries)) > > Accidentally I found out that 'vale-ctl -n testif0' creates a artificial > interface, which is reported by ifconfig(8): > testif0: flags=3D8801 metric 0 mtu 1500 > options=3D80000 > ether 00:be:eb:8d:f8:00 > nd6 options=3D21 > > But I can't assign a IP address: 'ifconfig testif0 203.0.113.1/24' > ifconfig: ioctl (SIOCAIFADDR): Invalid argument > > I guess couldn't geti the picture of the netmap(4) world yet. > Probably, testif0 is available only in netmap(4) world, not in "host > world". > I'm assuming, because I found vale-ctl(-8)s "-h" switch. > Yes, those are the "persistent" VALE ports. They are a recent feature, and probably you don't need to use them if you are going to play with Virtual Machines and jails (see below). > > So another very little peace I'm aware of the netmap(4) world, is how to > attach physical interfaces to virtual switches: > '/usr/src/tools/tools/netmap/vale-ctl -a vale0:em1' > Now vale-ctl(-8) shows: > bdg_ctl [149] bridge:0 port:0 vale0:em1 > > /* > To share my experience: One cannot use any other than vale[[:digit:]] > for defining the on-demand to be created virtual switch instance, so > e.g. "vale-ctl -a vale-test:em1" doesn't work, although found in > netmap(4) man page in FreeBSD-11: > =C2=BBvaleXXX:YYY (arbitrary XXX and YYY) > the file descriptor is bound to port YYY of a VALE switch called > XXX, both dynamically created if necessary. The string cannot > exceed IFNAMSIZ characters, and YYY cannot be the name of any > existing OS network interface=C2=AB > > I was about to give up on netmap(4) investigations because I thought it > isn't production ready yet (in FreeBSD), since even andding the first > physical interface fails: '/usr/src/tools/tools/netmap/vale-ctl -a > vale-test:em1' > vale-test:em1: Invalid argument > > Probably accidentally I used vale[[:digit:]] instead and wondered whay > it suddenly works=E2=80=A6 > Correct, this seems to be an inconsistency between the manual and the implementation, we will fix the manual. > > To get back to vale-ctl(-8)s "-h" switch: > */ > > If I add a physical interface with -h instead of -a, the host's IP stack > doesn't get disconnected from the interface, so it's still usable by > host applications and vale-ctl(-8) lists one line more: > bdg_ctl [149] bridge:0 port:0 vale0:em1 > bdg_ctl [149] bridge:0 port:1 vale0:em1^ > So my assumption that netmap(4) lives decapsuled from the well known > FreeBSD IP world. > > > Now my question: > > How can I plug a jail's or vmm's artificial interface to a VALE virtual > switch, bridging frames to real-world via physical interfaces? > (the latter part should work with vale-ctl -h vale0:em1, but what > interface to use for jail(8) vnet.interface and how to create/attach?) > If you use bhyve/vmm, you can attach the VM TAP interface to the VALE switch, as you would do for "em1". Regarding jails, I don't know exactly how networking works there, but I guess epair(4) interface (or similar) are used. If this is the case, then you would have one end of the epair only visible in the jail, and the other end only visible in the "host"; then you could attach the host end to a VALE switch again with "vale-ctl -a". Unfortunately, the performance you would get in any case is not great, because TAP and epair interface do not have netmap "native support". Moreover, when using bhyve, you have to pay the cost of the emulation of the vtnet device, since each packet passes through this device (other than passing across netmap). However, consider the following: a consistent netmap update is going to happen in FreeBSD-CURRENT, in short. This is going to align the netmap code which is now in FreeBSD to the code on the official github repository ( https://github.com/luigirizzo/netmap). Among the new features, there is a new solution for bhyve networking, which will let you attach your bhyve VMs directly to a VALE switch, without paying additional overheads related to TAPs, epairs, and vtnet emulation. You can find additional information, code and performance numbers here: https://wiki.freebsd.org/SummerOfCode2016/PtnetDriverAndDeviceModel. Cheers, Vincenzo > > Thanks, > > -harry > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --=20 Vincenzo Maffione