Date: Thu, 13 Feb 2014 15:22:22 -0500 From: Allan Jude <freebsd@allanjude.com> To: freebsd-current@freebsd.org Subject: Re: ezjails, systat -ifstat, and multiple network cards Message-ID: <52FD297E.6040502@allanjude.com> In-Reply-To: <CAK6zN=1V3j0f4w8K3z_RseCKQdG-R8ys%2BQ4pYthMiyUUhhQRew@mail.gmail.com> References: <CAK6zN=1V3j0f4w8K3z_RseCKQdG-R8ys%2BQ4pYthMiyUUhhQRew@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Dl4s3nJ5GBKTH9P9w2qpfqCUmA0E6DLoB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2014-02-13 13:59, Preston Hagar wrote: > I have a server setup with FreeBSD-10.0-RELEASE. It has 3 Intel gigabi= t > network cards in it, em0, em1, and em2. I have multiple ezjails setup = that > run various things. >=20 > One jail, called db, runs a postgresql database. It was my intention t= o > give it em0 all to itself. The other jails and host machine should be= > going through em2. em1 currently isn't being used. >=20 > If I do an ifconfig, I see that em0 has the alias IP for my db jail and= em2 > has the alias IP for all other jails. All the jails respond to network= > traffic as expected and seemingly work fine. >=20 > The weird thing is when I do a systat -ifstat from the host, it should > essentially all traffic going through em0. Some of the jails that run = off > of em2 (as defined in their jail config files and seen in ifconfig) hav= e > large data transfers and/or are web servers with lots of photos. I hav= e > even tried to manually scp a large file out of a jail setup through em2= and > the numbers don't seem to budge. >=20 > If I do netstat -i -b -n -I and check em0 and em2, it seems to support= the > numbers shown by systat -ifstat. However, if I use trafshow or iftop (= both > of which require choosing one interface at a time), they both seem to > indicate the traffic flowing through the interfaces as I would expect. >=20 > So I was curious if anyone had seen something like this before or had a= ny > ideas of what is going on. I have net.fibs=3D2 set in /boot/loader.con= f, but > in all the jails I current have jail_name_fib=3D"" as I haven't got aro= und to > fullying setting up fibs. Is that perhaps the issue? Is there any way= to > determine with certainty which jail is using which interface short of > physically pulling a network cable and seeing what stops working? >=20 > Here are the relevant lines from my db (the one that should be on em0) > config: >=20 > export jail_db_hostname=3D"db" > export jail_db_ip=3D"em0|10.1.10.2" >=20 > From another jail on em2 called www: >=20 > export jail_www_hostname=3D"www" > export jail_www_ip=3D"em2|10.1.10.7" >=20 > from ifconfig >=20 > em0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu = 1500 > options=3D4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,= WOL_MAGIC,VLAN_HWTSO> > ether 08:60:6e:13:94:06 > inet 10.1.1.4 netmask 0xffff0000 broadcast 10.1.255.255 > inet6 fe80::a60:6eff:fe13:9406%em0 prefixlen 64 scopeid 0x1 > inet 10.1.10.2 netmask 0xffffffff broadcast 10.1.10.2 > nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect (1000baseT <full-duplex>) > status: active >=20 > em2: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu = 1500 > options=3D4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,= WOL_MAGIC,VLAN_HWTSO> > ether 68:05:ca:13:74:2a > inet 10.1.1.2 netmask 0xffff0000 broadcast 10.1.255.255 > inet6 fe80::6a05:caff:fe13:742a%em2 prefixlen 64 scopeid 0x3 > inet 10.1.10.3 netmask 0xffffffff broadcast 10.1.10.3 > inet 10.1.10.1 netmask 0xffffffff broadcast 10.1.10.1 > inet 10.1.10.8 netmask 0xffffffff broadcast 10.1.10.8 > inet 10.1.10.10 netmask 0xffffffff broadcast 10.1.10.10 > inet 10.1.10.4 netmask 0xffffffff broadcast 10.1.10.4 > inet 10.1.10.9 netmask 0xffffffff broadcast 10.1.10.9 > inet 10.1.10.7 netmask 0xffffffff broadcast 10.1.10.7 > nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect (1000baseT <full-duplex>) > status: active >=20 >=20 > Let me know if any more detail would be helpful or if you have any idea= s of > things to check. >=20 > Thanks, >=20 > Preston > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.o= rg" >=20 All traffic going out from the jails will using the routing table from the host system. The routing table will use the network card that is in the same subnet as your default gateway to route the traffic to the internet. In your case, I would imagine this is 10.1.1.4/16 (and 10.1.1.2/16). 'netstat -rn' will tell the tale, but I imagine it is whichever was added first. If you want to have separate routing tables per jail, you'd have to either use FIBs, and set the jails to use the different FIBs, or use VNET jails and have a routing table in each jail. --=20 Allan Jude --Dl4s3nJ5GBKTH9P9w2qpfqCUmA0E6DLoB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJS/SmBAAoJEJrBFpNRJZKfITEQAIfSKmFLr+VkRgmHob01tVqv c3ZecjqnRh9Co+GiqiNvnidnMFN986YO9KTr85/rpVK+AjD/rUn5K///DEGz++Go oFUbzCQ56BEbQi4QgGZfVc32LsYt9L97GeXuy9ENx29HTR0247acMhz1+JhmMnVz 1tsJYlJ2XRDYwkXpNCiRUV/RV1REmJvz7LjXx9ZC1dCyCSNhPPo4X9fdKdgoHAWJ SDMJ5KbRDz/AmCQNEwf/wXA7FYE694WOO+bsF7F27mcVfWpFjTYbnOue4jY+3qjh Pg+/BBPa9eX1SfuYy9D89oV5ABbtshhcjLyUhl7QX3dEdhZPSlLePAuXN3Pv84MZ 90mPbeeY5B8bbaC6hO5vocXXVHWyfaWb1XQQzKIfv34W+FiketDWlGhWm39IpdCi NJ0OZL8ji/M6PQCCDLifSufwrrMbEHr3G3hR6vNCsgkgxRhDKH0C5r4jgOIgmfrG Z6ezfjz59HxI65lsi34lPt4qIXAtivByL9baaQdmrwwshRTcXyB7otoAqJmA1ehn 8Z0XJx645UKVhKdZxkX4PBvE0Sy9rFE1D1Iu1aMKT7P7NmGUJy3amLETYtvFGt8z 5m75RrDTVwXcEZU1vlAJGnOcd5ispkA5AzyAlNKJDNSI8i8Gz7Rwns5EVwMEGIBc tBhUBIFt5O8PA8eOLJ+b =XSlw -----END PGP SIGNATURE----- --Dl4s3nJ5GBKTH9P9w2qpfqCUmA0E6DLoB--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?52FD297E.6040502>