Date: Mon, 17 Feb 2014 11:21:33 -0600 From: Preston Hagar <prestonh@gmail.com> To: Allan Jude <freebsd@allanjude.com> Cc: freebsd-current@freebsd.org Subject: Re: ezjails, systat -ifstat, and multiple network cards Message-ID: <CAK6zN=3xw08D%2BwX-tPGO%2BAgKzMTAzdrhJUAuhf0vM%2BTEY1E_ng@mail.gmail.com> In-Reply-To: <52FD297E.6040502@allanjude.com> References: <CAK6zN=1V3j0f4w8K3z_RseCKQdG-R8ys%2BQ4pYthMiyUUhhQRew@mail.gmail.com> <52FD297E.6040502@allanjude.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Feb 13, 2014 at 2:22 PM, Allan Jude <freebsd@allanjude.com> wrote: > On 2014-02-13 13:59, Preston Hagar wrote: > > I have a server setup with FreeBSD-10.0-RELEASE. It has 3 Intel gigabit > > network cards in it, em0, em1, and em2. I have multiple ezjails setup > that > > run various things. > > > > One jail, called db, runs a postgresql database. It was my intention to > > give it em0 all to itself. The other jails and host machine should be > > going through em2. em1 currently isn't being used. > > > > If I do an ifconfig, I see that em0 has the alias IP for my db jail and > em2 > > has the alias IP for all other jails. All the jails respond to network > > traffic as expected and seemingly work fine. > > > > The weird thing is when I do a systat -ifstat from the host, it should > > essentially all traffic going through em0. Some of the jails that run > off > > of em2 (as defined in their jail config files and seen in ifconfig) have > > large data transfers and/or are web servers with lots of photos. I have > > even tried to manually scp a large file out of a jail setup through em2 > and > > the numbers don't seem to budge. > > > > If I do netstat -i -b -n -I and check em0 and em2, it seems to support > the > > numbers shown by systat -ifstat. However, if I use trafshow or iftop > (both > > of which require choosing one interface at a time), they both seem to > > indicate the traffic flowing through the interfaces as I would expect. > > > > So I was curious if anyone had seen something like this before or had any > > ideas of what is going on. I have net.fibs=2 set in /boot/loader.conf, > but > > in all the jails I current have jail_name_fib="" as I haven't got around > to > > fullying setting up fibs. Is that perhaps the issue? Is there any way > to > > determine with certainty which jail is using which interface short of > > physically pulling a network cable and seeing what stops working? > > > > Here are the relevant lines from my db (the one that should be on em0) > > config: > > > > export jail_db_hostname="db" > > export jail_db_ip="em0|10.1.10.2" > > > > From another jail on em2 called www: > > > > export jail_www_hostname="www" > > export jail_www_ip="em2|10.1.10.7" > > > > from ifconfig > > > > em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > > > options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO> > > ether 08:60:6e:13:94:06 > > inet 10.1.1.4 netmask 0xffff0000 broadcast 10.1.255.255 > > inet6 fe80::a60:6eff:fe13:9406%em0 prefixlen 64 scopeid 0x1 > > inet 10.1.10.2 netmask 0xffffffff broadcast 10.1.10.2 > > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > > media: Ethernet autoselect (1000baseT <full-duplex>) > > status: active > > > > em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > > > options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO> > > ether 68:05:ca:13:74:2a > > inet 10.1.1.2 netmask 0xffff0000 broadcast 10.1.255.255 > > inet6 fe80::6a05:caff:fe13:742a%em2 prefixlen 64 scopeid 0x3 > > inet 10.1.10.3 netmask 0xffffffff broadcast 10.1.10.3 > > inet 10.1.10.1 netmask 0xffffffff broadcast 10.1.10.1 > > inet 10.1.10.8 netmask 0xffffffff broadcast 10.1.10.8 > > inet 10.1.10.10 netmask 0xffffffff broadcast 10.1.10.10 > > inet 10.1.10.4 netmask 0xffffffff broadcast 10.1.10.4 > > inet 10.1.10.9 netmask 0xffffffff broadcast 10.1.10.9 > > inet 10.1.10.7 netmask 0xffffffff broadcast 10.1.10.7 > > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > > media: Ethernet autoselect (1000baseT <full-duplex>) > > status: active > > > > > > Let me know if any more detail would be helpful or if you have any ideas > of > > things to check. > > > > Thanks, > > > > Preston > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscribe@freebsd.org" > > > > All traffic going out from the jails will using the routing table from > the host system. The routing table will use the network card that is in > the same subnet as your default gateway to route the traffic to the > internet. > > In your case, I would imagine this is 10.1.1.4/16 (and 10.1.1.2/16). > > 'netstat -rn' will tell the tale, but I imagine it is whichever was > added first. > > If you want to have separate routing tables per jail, you'd have to > either use FIBs, and set the jails to use the different FIBs, or use > VNET jails and have a routing table in each jail. > > -- > Allan Jude > > Makes sense, thank you. I'll setup the FIBs. Preston
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAK6zN=3xw08D%2BwX-tPGO%2BAgKzMTAzdrhJUAuhf0vM%2BTEY1E_ng>