Date: Sat, 1 Jun 2013 03:26:16 +0000 From: "Teske, Devin" <Devin.Teske@fisglobal.com> To: Pietro Paolini <pulsarpietro@aol.com> Cc: Devin Teske <dteske@freebsd.org>, FreeBSD Questions <freebsd-questions@freebsd.org> Subject: Re: VIMAGE Message-ID: <13CA24D6AB415D428143D44749F57D7201F6BCEB@ltcfiswmsgmb21> In-Reply-To: <13CA24D6AB415D428143D44749F57D7201F6B5F0@ltcfiswmsgmb21> References: <DB90C1DC-66E4-4429-A888-44F4F9E4B98B@aol.com> <13CA24D6AB415D428143D44749F57D7201F68CBD@ltcfiswmsgmb21> <DA96E7A7-C419-4C73-A27B-D02BAB2CBE4E@aol.com> <13CA24D6AB415D428143D44749F57D7201F6B5F0@ltcfiswmsgmb21>
next in thread | previous in thread | raw e-mail | index | archive | help
On May 31, 2013, at 3:05 PM, Teske, Devin wrote: On May 31, 2013, at 1:48 AM, Pietro Paolini wrote: On May 30, 2013, at 6:25 PM, "Teske, Devin" <Devin.Teske@fisglobal.com<mail= to:Devin.Teske@fisglobal.com>> wrote: On May 30, 2013, at 3:35 AM, Pietro Paolini wrote: Hello all, I am a new bye on the FreeBSD and I am looking at the VIMAGE features exper= iencing some problems. I added the options : VIMAGE if_bridge and I removed STCP then I recompiled my kernel and install it. After that, following this tutorial http://imunes.tel.fer.hr/virtnet/eurobs= dcon07_tutorial.pdf I tried the "Exercise 2" which consist on the following commands: vimage -c n1 vimage -c n2 ngctl mkpeer efface ether ether ngctl mkpeer efface ether ether Don't you just love autocorrect? (does the same thing to me=85 turns "eifac= e" into "efface") ngctl mkpeer em0: bridge lower link0 Looks good. ngctl name em0:lower bridge0 I usually do my "connect" before the "name"=85 but shouldn't matter. Should= work all the same. ngctl connect em0: bridge0: upper link1 This looks wrong to me. I'd expect: ngctl connect em0: bridge0:lower upper link1 Many thanks for the answer Devin, when I try to use that last command I receive: ngctl connect em0: bridge0:lower upper link1 ngctl: send msg: Invalid argument What's wrong ? Let's start from scratch on a freshly booted box=85 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl ls -l [sudo] Password: There are 4 total nodes: Name: em0 Type: ether ID: 00000002 Num hooks: 0 Name: em1 Type: ether ID: 00000003 Num hooks: 0 Name: ngctl1719 Type: socket ID: 00000004 Num hooks: 0 Name: msk0 Type: ether ID: 00000001 Num hooks: 0 Ok=85 we have an "ether" type node for each of our physical adapters (these= are provided by ng_ether(4); you didn't have to do anything to get these n= odes). We also have a single "socket" type node. This is the "ngctl" connection to= the netgraph subsystem (you can learn more by reading ng_socket(4)). Here's the corresponding hardware behind em0, em1, and msk0: =3D=3D=3D dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ grep '\= (em\|e1000phy\|mskc\?\)[[:digit:]]' /var/run/dmesg.boot mskc0: <Marvell Yukon 88E8050 Gigabit Ethernet> port 0xdc00-0xdcff mem 0xfc= ffc000-0xfcffffff irq 16 at device 0.0 on pci5 msk0: <Marvell Technology Group Ltd. Yukon EC Id 0xb6 Rev 0x02> on mskc0 msk0: Ethernet address: xx:xx:xx:xx:xx:xx miibus0: <MII bus> on msk0 e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 0 on miibus0 e1000phy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT= , 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto mskc0: [ITHREAD] em0: <Intel(R) PRO/1000 Legacy Network Connection 1.0.3> port 0xec80-0xecbf= mem 0xfebe0000-0xfebfffff irq 16 at device 4.0 on pci7 em0: [FILTER] em0: Ethernet address: xx:xx:xx:xx:xx:xx em1: <Intel(R) PRO/1000 Legacy Network Connection 1.0.3> port 0xec00-0xec3f= mem 0xfeba0000-0xfebbffff,0xfeb80000-0xfeb9ffff irq 18 at device 6.0 on pc= i7 em1: [FILTER] em1: Ethernet address: xx:xx:xx:xx:xx:xx em0: link state changed to UP =3D=3D=3D Next, let's make a bridge (think of it as a big software switch that we're = going to hook a bunch of interfaces; created, physical, or otherwise). Since I'm doing this over an SSH connection (a mistake I made earlier today= ), I'm not going to touch em0 (the adapter my SSH connection is using). Cre= ating the bridge on an actively configured PHY will knock it off the net. T= his is not to say you can't have an active configuration on a bridged inter= face=85 just that the creation of the bridge (something you should only do = once each time you boot) will disrupt an active connection. So=85 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl mkpeer em1: bridge lower link0 NOTE: No output =3D=3D Success. =3D=3D=3D Now let's look at our handiwork=85 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl info em1:lower Name: <unnamed> Type: bridge ID: 00000007 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link0 em1 ether 00000003 lower Ok, we see that the lower peer hook of the em1 ether-node goes off to somet= hing named "link0". To see where link0 is off-to=85 we need a full listing (back to "ngctl ls -= l"). dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl ls -l There are 5 total nodes: Name: <unnamed> Type: bridge ID: 00000007 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link0 em1 ether 00000003 lower Name: em0 Type: ether ID: 00000002 Num hooks: 0 Name: em1 Type: ether ID: 00000003 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- lower <unnamed> bridge 00000007 link0 Name: ngctl1762 Type: socket ID: 0000000b Num hooks: 0 Name: msk0 Type: ether ID: 00000001 Num hooks: 0 Matching "link0" in the first column to "link0" in the last-column, we can = see that this lower-link0 is to a bridge (with no name). NOTE: When you're digesting the above output=85 it helps to imagine whitesp= ace in between the nodes with their respective hooks and other nodes. Futur= e pastes below will introduce such whitespace to make it easier to read. =3D=3D=3D Right now, the only way to refer to the bridge is by way of "em1:lower" (be= cause we created the bridge right on the lower hook of the em1 ether-node). At this point, let's talk about naming. Giving our bridge a name is entirel= y optional, but greatly clarifies the output of both "ngctl ls -l" and "ngc= tl dot". dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl name em1:lower em1bridge dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl ls -l There are 5 total nodes: Name: em0 Type: ether ID: 00000002 Num hooks: 0 Name: em1 Type: ether ID: 00000003 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- lower em1bridge bridge 00000007 link0 Name: ngctl1831 Type: socket ID: 0000001a Num hooks: 0 Name: em1bridge Type: bridge ID: 00000007 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link0 em1 ether 00000003 lower Name: msk0 Type: ether ID: 00000001 Num hooks: 0 The new "em1bridge" name acts as an alias to "em1:lower" in future ngctl co= mmands. For example, "ngctl info em1:lower" and "ngctl info em1bridge" can = now be used interchangeably and produce the same results. dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl info em1bridge: Name: em1bridge Type: bridge ID: 00000007 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link0 em1 ether 00000003 lower dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl info em1:lower Name: em1bridge Type: bridge ID: 00000007 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link0 em1 ether 00000003 lower =3D=3D=3D We're not done with the bridge yet. Because we foresee the possibility that= it might be nice to be able to communicate with the jail that we're going = to later hook into this bridge=85 we should hook the physical adapter's "up= per" hook into the bridge. If you don't do this, you won't be able to (for example) ping a jail from t= he host where the host has only the PHY and the jail has only a (yet uncrea= ted) eiface. Regardless of the fact that the bridge uses the PHY and the ja= il uses the bridge, to communicate with an IP that is configured on the bas= e host, you must hook the upper. dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl connect em1: em1:lower upper link1 If you want to use the alias I set up earlier (of "em1bridge") that works t= oo (just don't forget the colon at the end of the alias): dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl connect em1: em1bridge: upper link1 Here's the results: dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl ls -l There are 5 total nodes: Name: em0 Type: ether ID: 00000002 Num hooks: 0 Name: em1 Type: ether ID: 00000003 Num hooks: 2 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- upper em1bridge bridge 0000002a link1 lower em1bridge bridge 0000002a link0 Name: ngctl1874 Type: socket ID: 00000030 Num hooks: 0 Name: em1bridge Type: bridge ID: 0000002a Num hooks: 2 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link1 em1 ether 00000003 upper link0 em1 ether 00000003 lower Name: msk0 Type: ether ID: 00000001 Num hooks: 0 NOTE: Some of the Peer ID's have changed, because I wanted to test that the= alias could be used; I used "sudo ngctl shutdown em1bridge:" and re-execut= ed up to the point where I connect the em1:upper into the bridge=85 except = this time using the alias of "em1bridge" instead of "em1:lower" (indeed, yo= u can use them interchangeably). =3D=3D=3D Ok=85 We've now done the hard part=85 which was to create and configure a b= ridge that is usable by any new nodes we connect to it and also (if you hoo= ked the upper portion of em1 back into its own lower which is acting as the= bridge) the base machine can communicate with any of the forth-coming jail= s (if on the same subnet at least). There's an easy step that shouldn't be skipped though=85 Before you can truly use this bridge with any other interfaces=85 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo if= config em1 up dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl msg em1: setpromisc 1 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl msg em1: setautosrc 0 A bridge cannot send packets out if the interface is down. A bridge cannot work properly without promiscuous mode. A bridge cannot send out packets for different addresses unless you turn of= f "setautosrc" =3D=3D=3D Let's create our first virtual NIC and connect it to the bridge. dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl mkpeer em1bridge: eiface link2 ether This command did two things. It created a new "eiface" node (see ng_eiface(= 4)), and connected it to the bridge. Let's have a look: dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl ls -l There are 6 total nodes: Name: em0 Type: ether ID: 00000002 Num hooks: 0 Name: em1 Type: ether ID: 00000003 Num hooks: 2 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- upper em1bridge bridge 0000002a link1 lower em1bridge bridge 0000002a link0 Name: ngeth0 Type: eiface ID: 00000035 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- ether em1bridge bridge 0000002a link2 Name: ngctl2800 Type: socket ID: 00000036 Num hooks: 0 Name: em1bridge Type: bridge ID: 0000002a Num hooks: 3 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link2 ngeth0 eiface 00000035 ether link1 em1 ether 00000003 upper link0 em1 ether 00000003 lower Name: msk0 Type: ether ID: 00000001 Num hooks: 0 The list of hooks for our bridge (em1bridge) is growing, and now we see a n= ew node (ngeth0) with one hook into that bridge. =3D=3D=3D ASIDE: If you wanted to script this=85 here's how you can test for an unuse= d link: Right now, we have link0, link1, and link2 for the bridge. If a link exists= for a bridge, the following command will return some info about the link a= nd return success (whereas if the link does not exist, the command will ret= urn an error and exit with error-status): dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl msg em1bridge: getstats 0 Rec'd response "getstats" (4) from "[2a]:": Args: {} dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl msg em1bridge: getstats 1 Rec'd response "getstats" (4) from "[2a]:": Args: {} dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl msg em1bridge: getstats 2 Rec'd response "getstats" (4) from "[2a]:": Args: {} dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl msg em1bridge: getstats 3 ngctl: send msg: Socket is not connected dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl msg em1bridge: getstats 4 ngctl: send msg: Socket is not connected dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl msg em1bridge: getstats 5 ngctl: send msg: Socket is not connected As you can see from the above output=85 we get errors for link3, link4, and= link5, because they don't exist. Naturally, testing $? exit status after e= ach of these commands would show how this can be scripted (HINT: throw stdo= ut/stderr to /dev/null and test $?). =3D=3D=3D At this point=85 you say "ifconfig": dteske@oos0a.lbxrich.vicor.com<mailto:dteske@oos0a.lbxrich.vicor.com> ~ $ i= fconfig msk0: flags=3D8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=3Dc011a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,VLAN_HWTSO,LINK= STATE> ether xx:xx:xx:xx:xx:xx media: Ethernet autoselect em0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=3D209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WO= L_MAGIC> ether xx:xx:xx:xx:xx:xx inet xx.xx.xx.xx netmask 0xffffff80 broadcast xx.xx.xx.xx media: Ethernet autoselect (1000baseT <full-duplex>) status: active em1: flags=3D8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 = mtu 1500 options=3D209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WO= L_MAGIC> ether xx:xx:xx:xx:xx:xx media: Ethernet autoselect status: no carrier ipfw0: flags=3D8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536 lo0: flags=3D8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=3D3<RXCSUM,TXCSUM> inet 127.0.0.1 netmask 0xff000000 ngeth0: flags=3D8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 00:00:00:00:00:00 =3D=3D=3D Ok, there are two problems with the network interface. 1. It has a NULL MAC address (00:00:00:00:00:00). Good luck communicating o= n the Internet (remember, we disabled setautosrc -- we intend to make up a = MAC address that is unique). 2. The name leaves something to be desired (if we're going to use this with= a vimage jail, it would be nice if the interface had the jail name in it, = so that when you do an "ngctl ls -l" or an "ngctl dot" =85 you're going to = see the jail name so it becomes clear which jails are hooked to which PHY's= through which bridges). =3D=3D=3D Let's tackle the easier one first=85 let's rename this new interface. You and I already know that this interface that we want to rename is "ngeth= 0"=85 but you can actually extract the name from the link in the bridge. dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl show -n em1bridge:link2 Name: ngeth0 Type: eiface ID: 00000035 Num hooks: 1 First, we rename it in netgraph (this does not affect the output of ifconfi= g -- and again, we do this to make "ngctl ls -l" and "ngctl dot" more palat= able): dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl name em1bridge:link2 ng0_myjail dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl ls -l There are 6 total nodes: Name: em0 Type: ether ID: 00000002 Num hooks: 0 Name: em1 Type: ether ID: 00000003 Num hooks: 2 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- upper em1bridge bridge 0000002a link1 lower em1bridge bridge 0000002a link0 Name: ngctl2843 Type: socket ID: 00000046 Num hooks: 0 Name: ng0_myjail Type: eiface ID: 00000035 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- ether em1bridge bridge 0000002a link2 Name: em1bridge Type: bridge ID: 0000002a Num hooks: 3 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link2 ng0_myjail eiface 00000035 ether link1 em1 ether 00000003 upper link0 em1 ether 00000003 lower Name: msk0 Type: ether ID: 00000001 Num hooks: 0 Looking good. However, ifconfig hasn't changed=85 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ ifconfig ... ngeth0: flags=3D8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 00:00:00:00:00:00 We want to rename the interface with ifconfig for a different reason. We renamed the interface with netgraph earlier so that netgraph outputs wou= ld be nice and easy to digest. This time, we rename with ifconfig so that we can layer jails onto the same= rootdir. The naming convention (which is the same naming convention I use for renami= ng on the netgraph side) is: ng#_name The # always starts at zero for each jail where "name" is the name of the j= ail. Again=85 I use this scheme so that I can layer jails onto the same root-dir= ; /etc/rc.conf is then populated with things like: ifconfig_ng0_myjail=3D... ifconfig_ng0_myrouter=3D... ifconfig_ng1_myrouter=3D... ifconfig_ng0_anotherjail=3D... So that when you say "service netif start" inside the vnet jail=85 it appli= es the right settings. So=85 we rename with ifconfig: dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo if= config ngeth0 name ng0_myjail dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ ifconfig ... ng0_myjail: flags=3D8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 00:00:00:00:00:00 =3D=3D=3D We're almost ready to shove this interface into a jail (which we haven't cr= eated yet). But=85 we come back to that NULL MAC address. NOTE: Forming your own MAC address, or even coming up with your own formula= should not be taken lightly. Here's a formula I use (which is based on several RFC's for MAC address for= mation): NOTE: In this context, ${_bridge} is em1 and $LINKNUM is 2 # Set the MAC address of the new interface # using a sensible algorithm to prevent # conflicts on the network. # # MAC LP:LL:LB:BB:BB:BB # P 2, 6, A, or E but usually 2 # NOTE: Indicates "privately administered" = MAC # L ng_bridge(4) link number (1-65535) # B Same as bridged interface # _bridge_ether=3D$( ifconfig ${_bridge} ethe= r | awk '/ether/{print $2}' ) _ether_devid=3D"${_bridge_ether#??:??:?}" n=3D$LINKNUM _quad=3D$(($n & 15)) case "${_quad}" in 10) _quad=3Da;; 11) _quad=3Db;; 12) _quad= =3Dc;; 13) _quad=3Dd;; 14) _quad=3De;; 15) _quad= =3Df;; esac _ether_devid=3D":${_quad}${_ether_devid}" n=3D$(($n >> 4)) _quad=3D$(($n & 15)) case "${_quad}" in 10) _quad=3Da;; 11) _quad=3Db;; 12) _quad= =3Dc;; 13) _quad=3Dd;; 14) _quad=3De;; 15) _quad= =3Df;; esac _ether_devid=3D"${_quad}${_ether_devid}" n=3D$(($n >> 4)) _quad=3D$(($n & 15)) case "${_quad}" in 10) _quad=3Da;; 11) _quad=3Db;; 12) _quad= =3Dc;; 13) _quad=3Dd;; 14) _quad=3De;; 15) _quad= =3Df;; esac _ether_devid=3D"2:${_quad}${_ether_devid}" n=3D$(($n >> 4)) _quad=3D$(($n & 15)) case "${_quad}" in 10) _quad=3Da;; 11) _quad=3Db;; 12) _quad= =3Dc;; 13) _quad=3Dd;; 14) _quad=3De;; 15) _quad= =3Df;; esac _ether_devid=3D"${_quad}${_ether_devid}" n=3D$(($n >> 4)) After which=85 ${_ether_devid} holds a properly formed MAC address that ca= n (in every case I've tested) "get out". Here's what I do to set it: ifconfig ng0_myjail ether "${_ether_devid}" Here's an example of how the MAC address was translated from the physical a= dapter to the ng_eiface(4) interface: dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ ifconfi= g em1; ifconfig ng0_myjail em1: flags=3D8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 = mtu 1500 options=3D209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC> ether 00:0e:0c:ab:1b:76 media: Ethernet autoselect status: no carrier ng0_myjail: flags=3D8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 02:00:2c:ab:1b:76 =3D=3D=3D OK=85 we're now ready to shove that interface into a vimage jail. But=85 First we need a vimage jail. (this is not a tutorial on how to create, mana= ge, build, or do anything else with jails, vimage-jails, or vps-jails *othe= r* than give it a netgraph based interface) I'm going to use my existing base machine as a fake jail (by pointing my ja= il's rootdir at "/"). NOTE: Certain sysctl's have to be set appropriately before you fire up the = jail to make this vimage jail able to do "more" on the net. dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo sy= sctl security.jail.set_hostname_allowed=3D1 security.jail.sysvipc_allowed= =3D1 security.jail.socket_unixiproute_only=3D1 security.jail.set_hostname_allowed: 1 -> 1 security.jail.sysvipc_allowed: 1 -> 1 security.jail.socket_unixiproute_only: 0 -> 1 NOTE: Unless you intend to reboot to restore the defaults later=85 you migh= t want to take down those previous values for restoration *after* we fire u= p the "vimage" jail. dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ja= il -i -c vnet name=3Dmyjail host.hostname=3Dmyjail path=3D/ persist 1 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ jls JID IP Address Hostname Path 1 - myjail / OK=85 we have a running jail (with the vnet property, making it a "vimage" = jail -- which can accept network interfaces). =3D=3D=3D Right now our jail has no network interfaces (well, it has an unconfigured = lo0). dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo je= xec myjail ifconfig lo0: flags=3D8008<LOOPBACK,MULTICAST> metric 0 mtu 16384 options=3D3<RXCSUM,TXCSUM> So let's pass the netgraph created interface into the jail=85 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo if= config ng0_myjail vnet 1 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo je= xec myjail ifconfig lo0: flags=3D8008<LOOPBACK,MULTICAST> metric 0 mtu 16384 options=3D3<RXCSUM,TXCSUM> ng0_myjail: flags=3D8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 02:00:2c:ab:1b:76 Sweet! =3D=3D=3D Almost there=85 Let's go into /etc/rc.conf, give it an IP, and start the network=85 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo sy= src ifconfig_ng0_myjail=3D"inet 192.168.1.1 netmask 255.255.255.0" /etc/rc.conf: ifconfig_ng0_myjail: -> inet 192.168.1.1 netmask 255.255.255= .0 dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ grep ng= 0 /etc/rc.conf ifconfig_ng0_myjail=3D"inet 192.168.1.1 netmask 255.255.255.0" dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo je= xec myjail service netif start Starting Network: lo0 ng0_myjail. lo0: flags=3D8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=3D3<RXCSUM,TXCSUM> inet 127.0.0.1 netmask 0xff000000 ng0_myjail: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 m= tu 1500 ether 02:00:2c:ab:1b:76 inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 Now we're cookin' with gasoline! =3D=3D=3D Optionally go configure your base machine with an IP and have fun. A quick conclusion=85 Because we've built this all on top of netgraph=85 we can =85 graph it. dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl dot | dot -Tsvg -o netgraph-scu0a.svg I then uploaded the file to the web and here it is: http://druidbsd.sourceforge.net/download/netgraph-scu0a.svg You should compare this directly to the output of "ngctl ls -l": dteske@scu0a.jbsd.vicor.com<mailto:dteske@scu0a.jbsd.vicor.com> ~ $ sudo ng= ctl ls -l There are 6 total nodes: Name: em0 Type: ether ID: 00000002 Num hooks: 0 Name: em1 Type: ether ID: 00000003 Num hooks: 2 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- upper em1bridge bridge 0000002a link1 lower em1bridge bridge 0000002a link0 Name: ng0_myjail Type: eiface ID: 00000035 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- ether em1bridge bridge 0000002a link2 Name: em1bridge Type: bridge ID: 0000002a Num hooks: 3 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- link2 ng0_myjail eiface 00000035 ether link1 em1 ether 00000003 upper link0 em1 ether 00000003 lower Name: ngctl8676 Type: socket ID: 00000049 Num hooks: 0 Name: msk0 Type: ether ID: 00000001 Num hooks: 0 You'll notice that when you graph the layout with "ngctl dot", the nodes ar= e rendered as boxes displaying their "Peer Name" up top, their "Peer Type" = in the lower-left, and their "Peer ID" in the bottom-right. The edges from one node to another contains two octagons. These are the "Lo= cal hook" and "Peer hook". -- Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?13CA24D6AB415D428143D44749F57D7201F6BCEB>