Date: Mon, 17 Jan 2011 14:26:24 -0800 From: Pyun YongHyeon <pyunyh@gmail.com> To: Paul Schenkeveld <freebsd@psconsult.nl> Cc: freebsd-net@freebsd.org, freebsd-stable@freebsd.org, re@freebsd.org Subject: Re: sis(4) broken on 8.2 [Re: Carp seems completely broken on 8.2-RC2 and 8.2-PRERELEASE] Message-ID: <20110117222624.GB1210@michelle.cdnetworks.com> In-Reply-To: <20110117212947.GA1210@michelle.cdnetworks.com> References: <20110116124122.GA63358@psconsult.nl> <20110117195615.GA18423@psconsult.nl> <20110117212947.GA1210@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--0eh6TmSyL6TZE2Uz
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On Mon, Jan 17, 2011 at 01:29:47PM -0800, Pyun YongHyeon wrote:
> On Mon, Jan 17, 2011 at 08:56:15PM +0100, Paul Schenkeveld wrote:
> > On Sun, Jan 16, 2011 at 01:41:22PM +0100, Paul Schenkeveld wrote:
> > > Hi,
> > >
> > > Trying to upgrade two Soekris firewalls to 8-STABLE or 8.2-PRERELEASE
> > > it appears that carp doesn't work at all. I've set up carp like I've
> > > done on many firewall pairs before and they all work correctly. With
> > > google, nor in the mailing lists, I could find anything about changes
> > > in the way carp get configured but if I missed something I'd be happy
> > > to hear that it's my fault.
> > >
> > > Here's the setup:
> > >
> > > net5501
> > > test3
> > > 10.4.0.4/24
> > > |
> > > -------------+-------------
> > > | |
> > > net4801 net4801
> > > test1 test2
> > > sis4: 10.4.0.2/24 sis4: 10.4.0.3/24
> > > carp4:10.4.0.1/24 carp4:10.4.0.1/24
> > > | | | | | | | |
> > > | | | | | | | |
> > > sis[0-3] connected to other networks, see
> > > explanation below.
> > >
> > > When I ping from test3 to 10.4.0.1, I see the following traffic using
> > > tcpdump:
> > >
> > > test3 # tcpdump -e -n -i vr3 not vrrp
> > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> > > listening on vr3, link-type EN10MB (Ethernet), capture size 96 bytes
> > > 12:09:35.121831 00:00:24:c9:30:ff > ff:ff:ff:ff:ff:ff,
> > > ethertype ARP (0x0806), length 60:
> > > Request who-has 10.4.0.1 tell 10.4.0.4, length 46
> > > 12:09:35.122144 00:00:24:c3:49:91 > 00:00:24:c9:30:ff,
> > > ethertype ARP (0x0806), length 60:
> > > Reply 10.4.0.1 is-at 00:00:5e:00:01:68, length 46
> > > 12:09:35.122173 00:00:24:c9:30:ff > 00:00:5e:00:01:68,
> > > ethertype IPv4 (0x0800), length 98:
> > > 10.4.0.4 > 10.4.0.1: ICMP echo request,
> > > id 40482, seq 0, length 64
> > >
> > > test1 # tcpdump -e -n -i sis4 not vrrp
> > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> > > listening on sis4, link-type EN10MB (Ethernet), capture size 96 bytes
> > > 12:09:34.977570 00:00:24:c9:30:ff > ff:ff:ff:ff:ff:ff,
> > > ethertype ARP (0x0806), length 60:
> > > Request who-has 10.4.0.1 tell 10.4.0.4, length 46
> > > 12:09:34.977705 00:00:24:c3:49:91 > 00:00:24:c9:30:ff,
> > > ethertype ARP (0x0806), length 42:
> > > Reply 10.4.0.1 is-at 00:00:5e:00:01:68, length 28
> > >
> > > test2 # dump -e -n -i sis4 not vrrp
> > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> > > listening on sis4, link-type EN10MB (Ethernet), capture size 96 bytes
> > > 12:09:35.090050 00:00:24:c9:30:ff > ff:ff:ff:ff:ff:ff,
> > > ethertype ARP (0x0806), length 60:
> > > Request who-has 10.4.0.1 tell 10.4.0.4, length 46
> > >
> > > There is an ARP request which is replied to by the carp master (test).
> > > the ping to the carp address does not even appear on the sis4 interface
> > > of test1.
> > >
> > > This is the kernel config for test1 and test2:
> > >
> > > include GENERIC
> > > device carp
> > > makeoptions MODULES_OVERRIDE=""
> > >
> > > The relevant rc.conf bits:
> > >
> > > on test1
> > > hostname="test1"
> > > cloned_interfaces="carp1 carp2 carp3 carp4"
> > > ifconfig_sis0="xxx.xxx.xxx.41/26"
> > > ifconfig_sis1="10.1.0.2/24"
> > > ifconfig_sis2="10.2.0.2/24"
> > > ifconfig_sis3="10.3.0.2/24"
> > > ifconfig_sis4="10.4.0.2/24"
> > > ifconfig_carp1="10.1.0.1/24 vhid 101 pass abcd1234 advskew 0"
> > > ifconfig_carp2="10.2.0.1/24 vhid 102 pass abcd1234 advskew 0"
> > > ifconfig_carp3="10.3.0.1/24 vhid 103 pass abcd1234 advskew 0"
> > > ifconfig_carp4="10.4.0.1/24 vhid 104 pass abcd1234 advskew 0"
> > >
> > > on test2
> > > hostname="test2"
> > > cloned_interfaces="carp1 carp2 carp3 carp4"
> > > ifconfig_sis0="xxx.xxx.xxx.42/26"
> > > ifconfig_sis1="10.1.0.3/24"
> > > ifconfig_sis2="10.2.0.3/24"
> > > ifconfig_sis3="10.3.0.3/24"
> > > ifconfig_sis4="10.4.0.3/24"
> > > ifconfig_carp1="10.1.0.1/24 vhid 101 pass abcd1234 advskew 100"
> > > ifconfig_carp2="10.2.0.1/24 vhid 102 pass abcd1234 advskew 100"
> > > ifconfig_carp3="10.3.0.1/24 vhid 103 pass abcd1234 advskew 100"
> > > ifconfig_carp4="10.4.0.1/24 vhid 104 pass abcd1234 advskew 100"
> > >
> > > In /etc/sysctl.conf:
> > > net.inet.carp.preempt=1
> > >
> > > Ifconfig output:
> > >
> > > test1 # ifconfig sis4
> > > sis4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
> > > options=83808<VLAN_MTU,WOL_UCAST,WOL_MCAST,WOL_MAGIC,LINKSTATE>
> > > ether 00:00:24:c3:49:91
> > > inet 10.4.0.2 netmask 0xffffff00 broadcast 10.4.0.255
> > > media: Ethernet autoselect (100baseTX <full-duplex>)
> > > status: active
> > > test1 # ifconfig carp4
> > > carp4: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
> > > inet 10.4.0.1 netmask 0xffffff00
> > > carp: MASTER vhid 104 advbase 1 advskew 0
> > >
> > > test2 # ifconfig sis4
> > > sis4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
> > > options=83808<VLAN_MTU,WOL_UCAST,WOL_MCAST,WOL_MAGIC,LINKSTATE>
> > > ether 00:00:24:c3:49:7d
> > > inet 10.4.0.3 netmask 0xffffff00 broadcast 10.4.0.255
> > > media: Ethernet autoselect (100baseTX <full-duplex>)
> > > status: active
> > > test2 # ifconfig carp4
> > > carp4: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
> > > inet 10.4.0.1 netmask 0xffffff00
> > > carp: BACKUP vhid 104 advbase 1 advskew 100
> > >
> > > There are no packet filters in place, sis1, sis2 and sis3 are wired
> > > through cross-cables from test1 to test2, so no traffic there except for
> > > carp. The sis4 interfaces and vr3 of test3 are on a dumb switch with no
> > > other stuff connected.
> > >
> > > Setting net.inet.carp.log=7 does not result in any console/dmesg/messages
> > > output.
> > >
> > > I see carp traffic on sis4 which appears normal except that I don't
> > > understand the addrs(7): part but that used to be there on 8.0/8.1
> > > firewalls too:
> > >
> > > 12:26:52.387140 00:00:5e:00:01:68 > 01:00:5e:00:00:12,
> > > ethertype IPv4 (0x0800), length 70:
> > > (tos 0x10, ttl 255, id 61070, offset 0, flags [DF],
> > > proto VRRP (112), length 56)
> > > 10.4.0.2 > 224.0.0.18: VRRPv2, Advertisement,
> > > vrid 104, prio 0, authtype none, intvl 1s, length 36,
> > > addrs(7): 198.145.25.33,1.75.182.226,80.169.106.108,
> > > 170.107.157.42,147.165.174.125,42.254.15.27,182.184.82.166
> > >
> > > 12:26:53.387903 00:00:5e:00:01:68 > 01:00:5e:00:00:12,
> > > ethertype IPv4 (0x0800), length 70:
> > > (tos 0x10, ttl 255, id 61479, offset 0, flags [DF],
> > > proto VRRP (112), length 56)
> > > 10.4.0.2 > 224.0.0.18: VRRPv2, Advertisement,
> > > vrid 104, prio 0, authtype none, intvl 1s, length 36,
> > > addrs(7): 101.233.35.135,163.243.214.16,230.125.241.59,
> > > 123.57.190.52,104.246.131.251,255.69.201.65,61.158.20.122
> >
> > After taking apart two Soekris 4801 and replacing the lan1621 dual sis(4)
> > network card by an Intel PRO/100S (fxp) or a PRO/1000/MT (em) card, carp
> > works again on this interface.
> >
> > Apparently the problem is not with carp(4) but with sis(4).
> >
> > Since Soekris hardware (net45xx, net48xx, lan16xx) is quite popular for
> > firewalls and carp is quite often used together with this hardware, this
> > looks like a showstopper for 8.2 :-(
> >
>
> Since you didn't post dmesg output I'm not sure what kind of
> controller you have but I guess it would be NS8381[56]. I
> overhauled sis(4) to make it work on all architectures so one of
> change, probably r212119, could be cause of the issue. Due to lack
> of SiS controllers I didn't touch multicast handling part so some
> part of code still relies on old wrong behavior of driver.
> Would you try attached patch and let me know whether it makes any
> difference?
>
Hmm, unfortunately it seems the patch above may not work since NS
data sheet says that filter function should be disabled before
touching other bits in the register.
Try this one instead.
> > Regards,
> >
> > Paul Schenkeveld
--0eh6TmSyL6TZE2Uz
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="sis.promisc.diff2"
Index: sys/dev/sis/if_sis.c
===================================================================
--- sys/dev/sis/if_sis.c (revision 217490)
+++ sys/dev/sis/if_sis.c (working copy)
@@ -149,6 +149,9 @@
static int sis_newbuf(struct sis_softc *, struct sis_rxdesc *);
static int sis_resume(device_t);
static int sis_rxeof(struct sis_softc *);
+static void sis_rxfilter(struct sis_softc *);
+static void sis_rxfilter_ns(struct sis_softc *);
+static void sis_rxfilter_sis(struct sis_softc *);
static void sis_start(struct ifnet *);
static void sis_startl(struct ifnet *);
static void sis_stop(struct sis_softc *);
@@ -806,80 +809,117 @@
}
static void
-sis_setmulti_ns(struct sis_softc *sc)
+sis_rxfilter(struct sis_softc *sc)
{
+
+ SIS_LOCK_ASSERT(sc);
+
+ if (sc->sis_type == SIS_TYPE_83815)
+ sis_rxfilter_ns(sc);
+ else
+ sis_rxfilter_sis(sc);
+}
+
+static void
+sis_rxfilter_ns(struct sis_softc *sc)
+{
struct ifnet *ifp;
struct ifmultiaddr *ifma;
- uint32_t h = 0, i, filtsave;
+ uint32_t h, i, filter;
int bit, index;
ifp = sc->sis_ifp;
-
- if (ifp->if_flags & IFF_ALLMULTI || ifp->if_flags & IFF_PROMISC) {
- SIS_CLRBIT(sc, SIS_RXFILT_CTL, NS_RXFILTCTL_MCHASH);
- SIS_SETBIT(sc, SIS_RXFILT_CTL, SIS_RXFILTCTL_ALLMULTI);
- return;
+ filter = CSR_READ_4(sc, SIS_RXFILT_CTL);
+ if (filter & SIS_RXFILTCTL_ENABLE) {
+ /*
+ * Filter should be disabled to program other bits.
+ */
+ CSR_WRITE_4(sc, SIS_RXFILT_CTL, filter & ~SIS_RXFILTCTL_ENABLE);
+ CSR_READ_4(sc, SIS_RXFILT_CTL);
}
+ filter &= ~(NS_RXFILTCTL_ARP | NS_RXFILTCTL_PERFECT |
+ NS_RXFILTCTL_MCHASH | SIS_RXFILTCTL_ALLPHYS | SIS_RXFILTCTL_BROAD |
+ SIS_RXFILTCTL_ALLMULTI);
+ if (ifp->if_flags & IFF_BROADCAST)
+ filter |= SIS_RXFILTCTL_BROAD;
/*
- * We have to explicitly enable the multicast hash table
- * on the NatSemi chip if we want to use it, which we do.
+ * For the NatSemi chip, we have to explicitly enable the
+ * reception of ARP frames, as well as turn on the 'perfect
+ * match' filter where we store the station address, otherwise
+ * we won't receive unicasts meant for this host.
*/
- SIS_SETBIT(sc, SIS_RXFILT_CTL, NS_RXFILTCTL_MCHASH);
- SIS_CLRBIT(sc, SIS_RXFILT_CTL, SIS_RXFILTCTL_ALLMULTI);
+ filter |= NS_RXFILTCTL_ARP | NS_RXFILTCTL_PERFECT;
- filtsave = CSR_READ_4(sc, SIS_RXFILT_CTL);
+ if (ifp->if_flags & (IFF_ALLMULTI | IFF_PROMISC)) {
+ filter |= SIS_RXFILTCTL_ALLMULTI;
+ if (ifp->if_flags & IFF_PROMISC)
+ filter |= SIS_RXFILTCTL_ALLPHYS;
+ } else {
+ /*
+ * We have to explicitly enable the multicast hash table
+ * on the NatSemi chip if we want to use it, which we do.
+ */
+ filter |= NS_RXFILTCTL_MCHASH;
- /* first, zot all the existing hash bits */
- for (i = 0; i < 32; i++) {
- CSR_WRITE_4(sc, SIS_RXFILT_CTL, NS_FILTADDR_FMEM_LO + (i*2));
- CSR_WRITE_4(sc, SIS_RXFILT_DATA, 0);
- }
+ /* first, zot all the existing hash bits */
+ for (i = 0; i < 32; i++) {
+ CSR_WRITE_4(sc, SIS_RXFILT_CTL, NS_FILTADDR_FMEM_LO +
+ (i * 2));
+ CSR_WRITE_4(sc, SIS_RXFILT_DATA, 0);
+ }
- if_maddr_rlock(ifp);
- TAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) {
- if (ifma->ifma_addr->sa_family != AF_LINK)
- continue;
- h = sis_mchash(sc,
- LLADDR((struct sockaddr_dl *)ifma->ifma_addr));
- index = h >> 3;
- bit = h & 0x1F;
- CSR_WRITE_4(sc, SIS_RXFILT_CTL, NS_FILTADDR_FMEM_LO + index);
- if (bit > 0xF)
- bit -= 0x10;
- SIS_SETBIT(sc, SIS_RXFILT_DATA, (1 << bit));
+ if_maddr_rlock(ifp);
+ TAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) {
+ if (ifma->ifma_addr->sa_family != AF_LINK)
+ continue;
+ h = sis_mchash(sc,
+ LLADDR((struct sockaddr_dl *)ifma->ifma_addr));
+ index = h >> 3;
+ bit = h & 0x1F;
+ CSR_WRITE_4(sc, SIS_RXFILT_CTL, NS_FILTADDR_FMEM_LO +
+ index);
+ if (bit > 0xF)
+ bit -= 0x10;
+ SIS_SETBIT(sc, SIS_RXFILT_DATA, (1 << bit));
+ }
+ if_maddr_runlock(ifp);
}
- if_maddr_runlock(ifp);
- CSR_WRITE_4(sc, SIS_RXFILT_CTL, filtsave);
+ CSR_WRITE_4(sc, SIS_RXFILT_CTL, filter);
+ CSR_READ_4(sc, SIS_RXFILT_CTL);
}
static void
-sis_setmulti_sis(struct sis_softc *sc)
+sis_rxfilter_sis(struct sis_softc *sc)
{
struct ifnet *ifp;
struct ifmultiaddr *ifma;
- uint32_t h, i, n, ctl;
+ uint32_t filter, h, i, n;
uint16_t hashes[16];
ifp = sc->sis_ifp;
/* hash table size */
- if (sc->sis_rev >= SIS_REV_635 ||
- sc->sis_rev == SIS_REV_900B)
+ if (sc->sis_rev >= SIS_REV_635 || sc->sis_rev == SIS_REV_900B)
n = 16;
else
n = 8;
- ctl = CSR_READ_4(sc, SIS_RXFILT_CTL) & SIS_RXFILTCTL_ENABLE;
-
+ filter = CSR_READ_4(sc, SIS_RXFILT_CTL);
+ if (filter & SIS_RXFILTCTL_ENABLE) {
+ CSR_WRITE_4(sc, SIS_RXFILT_CTL, filter & ~SIS_RXFILT_CTL);
+ CSR_READ_4(sc, SIS_RXFILT_CTL);
+ }
+ filter &= ~(SIS_RXFILTCTL_ALLPHYS | SIS_RXFILTCTL_BROAD |
+ SIS_RXFILTCTL_ALLMULTI);
if (ifp->if_flags & IFF_BROADCAST)
- ctl |= SIS_RXFILTCTL_BROAD;
+ filter |= SIS_RXFILTCTL_BROAD;
- if (ifp->if_flags & IFF_ALLMULTI || ifp->if_flags & IFF_PROMISC) {
- ctl |= SIS_RXFILTCTL_ALLMULTI;
+ if (ifp->if_flags & (IFF_ALLMULTI | IFF_PROMISC)) {
+ filter |= SIS_RXFILTCTL_ALLMULTI;
if (ifp->if_flags & IFF_PROMISC)
- ctl |= SIS_RXFILTCTL_BROAD|SIS_RXFILTCTL_ALLPHYS;
+ filter |= SIS_RXFILTCTL_ALLPHYS;
for (i = 0; i < n; i++)
hashes[i] = ~0;
} else {
@@ -897,7 +937,7 @@
}
if_maddr_runlock(ifp);
if (i > n) {
- ctl |= SIS_RXFILTCTL_ALLMULTI;
+ filter |= SIS_RXFILTCTL_ALLMULTI;
for (i = 0; i < n; i++)
hashes[i] = ~0;
}
@@ -908,7 +948,8 @@
CSR_WRITE_4(sc, SIS_RXFILT_DATA, hashes[i]);
}
- CSR_WRITE_4(sc, SIS_RXFILT_CTL, ctl);
+ CSR_WRITE_4(sc, SIS_RXFILT_CTL, filter);
+ CSR_READ_4(sc, SIS_RXFILT_CTL);
}
static void
@@ -2104,41 +2145,7 @@
CSR_WRITE_4(sc, NS_PHY_PAGE, 0);
}
- /*
- * For the NatSemi chip, we have to explicitly enable the
- * reception of ARP frames, as well as turn on the 'perfect
- * match' filter where we store the station address, otherwise
- * we won't receive unicasts meant for this host.
- */
- if (sc->sis_type == SIS_TYPE_83815) {
- SIS_SETBIT(sc, SIS_RXFILT_CTL, NS_RXFILTCTL_ARP);
- SIS_SETBIT(sc, SIS_RXFILT_CTL, NS_RXFILTCTL_PERFECT);
- }
-
- /* If we want promiscuous mode, set the allframes bit. */
- if (ifp->if_flags & IFF_PROMISC) {
- SIS_SETBIT(sc, SIS_RXFILT_CTL, SIS_RXFILTCTL_ALLPHYS);
- } else {
- SIS_CLRBIT(sc, SIS_RXFILT_CTL, SIS_RXFILTCTL_ALLPHYS);
- }
-
- /*
- * Set the capture broadcast bit to capture broadcast frames.
- */
- if (ifp->if_flags & IFF_BROADCAST) {
- SIS_SETBIT(sc, SIS_RXFILT_CTL, SIS_RXFILTCTL_BROAD);
- } else {
- SIS_CLRBIT(sc, SIS_RXFILT_CTL, SIS_RXFILTCTL_BROAD);
- }
-
- /*
- * Load the multicast filter.
- */
- if (sc->sis_type == SIS_TYPE_83815)
- sis_setmulti_ns(sc);
- else
- sis_setmulti_sis(sc);
-
+ sis_rxfilter(sc);
/* Turn the receive filter on */
SIS_SETBIT(sc, SIS_RXFILT_CTL, SIS_RXFILTCTL_ENABLE);
@@ -2252,27 +2259,19 @@
if (ifp->if_flags & IFF_UP) {
if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0 &&
((ifp->if_flags ^ sc->sis_if_flags) &
- (IFF_PROMISC | IFF_ALLMULTI)) != 0) {
- if (sc->sis_type == SIS_TYPE_83815)
- sis_setmulti_ns(sc);
- else
- sis_setmulti_sis(sc);
- } else
+ (IFF_PROMISC | IFF_ALLMULTI)) != 0)
+ sis_rxfilter(sc);
+ else
sis_initl(sc);
- } else if (ifp->if_drv_flags & IFF_DRV_RUNNING) {
+ } else if (ifp->if_drv_flags & IFF_DRV_RUNNING)
sis_stop(sc);
- }
sc->sis_if_flags = ifp->if_flags;
SIS_UNLOCK(sc);
- error = 0;
break;
case SIOCADDMULTI:
case SIOCDELMULTI:
SIS_LOCK(sc);
- if (sc->sis_type == SIS_TYPE_83815)
- sis_setmulti_ns(sc);
- else
- sis_setmulti_sis(sc);
+ sis_rxfilter(sc);
SIS_UNLOCK(sc);
break;
case SIOCGIFMEDIA:
--0eh6TmSyL6TZE2Uz--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110117222624.GB1210>