Date: Sun, 4 Jan 2004 03:19:10 +1100 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: "Bruce A. Mah" <bmah@FreeBSD.org> Cc: freebsd-net@FreeBSD.org Subject: Re: bridge with access on both interfaces - reprise Message-ID: <Pine.BSF.3.96.1040103235008.24040A-100000@gaia.nimnet.asn.au> In-Reply-To: <20031225205212.GA64786@intruder.kitchenlab.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 25 Dec 2003, Bruce A. Mah sent me a useful Christmas present:
> In 4-STABLE, there's a bug that prevents ARP from working correctly on
> unnumbered bridge interfaces when bridging is enabled using the
> bridge.ko module. Basically, there are some checks in the ARP code
> that decide when to accept inbound ARP packets. These checks are a
> little different when the inbound interface is part of a bridge group.
> Some of these tests are conditional on the BRIDGE preprocessor symbol;
> this symbol gets defined if "options BRIDGE" is compiled into the
> kernel but not if you use the bridge.ko module. As a result, ARP
> packets on unnumbered interfaces get thrown away.
>
> The workaround for this problem is just to compile BRIDGE into the
> kernel. Manuel Kasper and I spent a few cycles working on this trying
> to make a m0n0wall box into a filtering bridge.
Your advice was of course right on, and I can't believe I've waited till
now to look over m0n0wall; dead cute. Running a kernel with 'options
BRIDGE' appears indeed to have solved the problem mentioned, and I can't
say the earlier struggle didn't have some educational value.
Only one noticeable issue is defying comprehension, concerning rwhod,
although I suspect that other UDP services communicating via broadcasts
might? have the same problem. I'll try to be succinct, more detailed
info and logging if this doesn't ring any bells for anyone ..
Test rig, all boxes running rwhod (denied to/from outside our net, OC)
and all within a.b.c.168/29 subnet currently, broadcast a.b.c.175
nuvo (4.5-RELEASE GENERIC) 'outside' proxy server for the exercise
ex0 a.b.c.169/29 ether :1d
|
ed1 no IP, ether :3d
blackstump (4.8-RELEASE with BRIDGE kernel, bridge + ipfw)
ed0 a.b.c.172/29 ether :a5
|
ed1 a.b.c.171/29 ether :db
gaia (gateway/router/firewall) -> ed0 a.b.c.d/28 -- local LAN
tun0 ppp -ddial
|
world
blackstump# sysctl -a|grep bridge
net.link.ether.bridge_cfg: ed0,ed1
net.link.ether.bridge: 1
net.link.ether.bridge_ipfw: 1
net.link.ether.bridge_ipfw_drop: 0
net.link.ether.bridge_ipfw_collisions: 0
Now rwho -a and ruptime on both gaia and nuvo, either side of the bridge
(which is working fine for everything else, afaik), show all three boxes
rwho data correctly:
gaia# rwho -a # 'inside' the bridge
smithi blackstump:ttyp1 Jan 3 23:16 :04
smithi gaia:ttyd2 Jan 3 23:11
smithi gaia:ttyp0 Jan 3 23:30
smithi nuvo:ttyp0 Jan 3 23:15
gaia# ruptime
blackstump up 14:01, 1 user, load 0.00, 0.00, 0.00
gaia up 66+12:25, 2 users, load 0.06, 0.04, 0.05
nuvo up 58+03:19, 1 user, load 0.00, 0.00, 0.00
tubi down 1+06:45
smithi on nuvo% ruptime # 'outside' the bridge, rwho -a ok too
blackstump up 13:55, 1 user, load 0.00, 0.00, 0.00
gaia up 66+12:19, 1 user, load 0.01, 0.03, 0.06
nuvo up 58+03:13, 1 user, load 0.00, 0.00, 0.00
tubi down 1+06:39
However, on the bridge box itself, rwho info for nuvo ('outside', on the
second bridged interface ed1) is simply never used, though tcpdump shows
nuvo's broadcast rwho packets in transit, viewing EITHER ed0 or ed1:
smithi on blackstump% ll -rt /var/rwho
total 4
-rw-r--r-- 1 daemon daemon 108 Jan 4 00:32 whod.gaia
-rw-r--r-- 1 daemon daemon 84 Jan 4 00:33 whod.blackstump
smithi on blackstump% rwho -a
smithi blackstump:ttyp1 Jan 3 23:16 :55
smithi gaia:ttyd2 Jan 3 23:11
smithi gaia:ttyp0 Jan 3 23:30
smithi on blackstump% ruptime
blackstump up 15:01, 1 user, load 0.00, 0.00, 0.00
gaia up 66+13:25, 2 users, load 0.10, 0.16, 0.11
blackstump# tcpdump -en -i ed0 not tcp port 22 # 'inside'
00:41:38.129758 0:aa:0:b7:6c:1d ff:ff:ff:ff:ff:ff 0800 126: a.b.c.169.513 > a.b.c.175.513: udp 84
[nb these being seen ok on the 'opposite' interface as well as below]
00:41:51.185141 0:80:48:9e:b:db ff:ff:ff:ff:ff:ff 0800 150: a.b.c.171.513 > a.b.c.175.513: udp 108
00:42:16.693267 52:54:5:e3:d9:a5 ff:ff:ff:ff:ff:ff 0800 126: a.b.c.172.513 > a.b.c.175.513: udp 84
00:42:16.693319 52:54:5:e3:d9:a5 ff:ff:ff:ff:ff:ff 0800 126: a.b.c.172.513 > a.b.c.175.513: udp 84
blackstump# tcpdump -en -i ed1 not tcp port 22 # 'outside'
tcpdump: WARNING: ed1: no IPv4 address assigned
00:54:16.744205 52:54:5:e3:d9:a5 ff:ff:ff:ff:ff:ff 0800 126: a.b.c.172.513 > a.b.c.175.513: udp 84
[from us, broadcast out our non-IP configured interface, ok]
00:54:17.681051 0:80:48:9e:b:db ff:ff:ff:ff:ff:ff 0806 60: arp who-has a.b.c.172 tell a.b.c.171
[is-at a.b.c.172 responses are only seen (appropriately) on ed0]
00:56:38.176140 0:aa:0:b7:6c:1d ff:ff:ff:ff:ff:ff 0800 126: a.b.c.169.513 > a.b.c.175.513: udp 84
[ie from the 'outside' box that is showing our rwho info, but that
we're not seeing - or at least accepting? from ed1]
00:56:51.391045 0:80:48:9e:b:db ff:ff:ff:ff:ff:ff 0800 150: a.b.c.171.513 > a.b.c.175.513: udp 108
00:57:16.756941 52:54:5:e3:d9:a5 ff:ff:ff:ff:ff:ff 0800 126: a.b.c.172.513 > a.b.c.175.513: udp 84
00:59:38.185421 0:aa:0:b7:6c:1d ff:ff:ff:ff:ff:ff 0800 126: a.b.c.169.513 > a.b.c.175.513: udp 84
00:59:51.432268 0:80:48:9e:b:db ff:ff:ff:ff:ff:ff 0800 150: a.b.c.171.513 > a.b.c.175.513: udp 108
01:00:16.769642 52:54:5:e3:d9:a5 ff:ff:ff:ff:ff:ff 0800 126: a.b.c.172.513 > a.b.c.175.513: udp 84
01:00:36.625568 0:80:48:9e:b:db ff:ff:ff:ff:ff:ff 0806 60: arp who-has a.b.c.169 tell a.b.c.171
01:00:36.625890 0:aa:0:b7:6c:1d 0:80:48:9e:b:db 0806 60: arp reply a.b.c.169 is-at 0:aa:0:b7:6c:1d
blackstump has specific firewall rules allowing (& counting) ournet/29
to ournet/29 513 bridged via either interface; happens anyway without fw
Any idea why the bridge box itself isn't seeing (or at least, accepting)
rwho data for the 'outside' box on the ed1 segment (which, as above, it
also sees broadcast on its 'inside' interface as well?) Same result when
it had an alias /32 IP assigned to ed1, both addresses being pingable.
> For more specifics, see src/sys/netinet/if_ether.c (grep for BRIDGE in
> this file).
Only as a last resort this time :) This box needs to be put into its
production environment this week, and its main function, bridging and
firewalling a satellite feed to a bunch of misconfigured and insecure
winXPs, seems to getting along just fine, so this is no show-stopper.
Thanks again Bruce, and thanks in advance to anyone who has a clue to
beat me with.
Cheers, Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.3.96.1040103235008.24040A-100000>