From owner-freebsd-net@FreeBSD.ORG Mon Feb 5 23:42:48 2007 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E098E16A400 for ; Mon, 5 Feb 2007 23:42:47 +0000 (UTC) (envelope-from oleg@lath.rinet.ru) Received: from lath.rinet.ru (lath.rinet.ru [195.54.192.90]) by mx1.freebsd.org (Postfix) with ESMTP id B4C2E13C467 for ; Mon, 5 Feb 2007 23:42:46 +0000 (UTC) (envelope-from oleg@lath.rinet.ru) Received: from lath.rinet.ru (localhost [127.0.0.1]) by lath.rinet.ru (8.13.8/8.13.8) with ESMTP id l15NS0VL047367 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 6 Feb 2007 02:28:00 +0300 (MSK) (envelope-from oleg@lath.rinet.ru) Received: (from oleg@localhost) by lath.rinet.ru (8.13.8/8.13.8/Submit) id l15NS0UR047366; Tue, 6 Feb 2007 02:28:00 +0300 (MSK) (envelope-from oleg) Date: Tue, 6 Feb 2007 02:28:00 +0300 From: Oleg Bulyzhin To: Robin Gruyters Message-ID: <20070205232800.GA45487@lath.rinet.ru> References: <20070125170532.c9c2374hkwk4oc4k@server.yirdis.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="k+w/mQv8wyuph6w0" Content-Disposition: inline In-Reply-To: <20070125170532.c9c2374hkwk4oc4k@server.yirdis.net> User-Agent: Mutt/1.5.13 (2006-08-11) Cc: freebsd-net@freebsd.org Subject: Re: [Fwd: Re: bge Ierr rate increase from 5.3R -> 6.1R] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Feb 2007 23:42:48 -0000 --k+w/mQv8wyuph6w0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Jan 25, 2007 at 05:05:32PM +0100, Robin Gruyters wrote: > On 13-Jan-2007 Oleg Bulyzhin wrote: > > > >> Could you please test attached patch? It should fix ierrs issue > >and should not > >> break link events (would be fine to test it: unplug/plug cable, > >try to change > >> media with ifconfig, change media on other end of wire). > >> > Hi ya, > > Just wondering will this patch/update be available for RELENG_6? I > have the same problems here with 3 of our servers. > > Here is dmesg from our development server: > > [...] > Copyright (c) 1992-2007 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 6.2-RELEASE #1: Mon Jan 15 15:34:19 CET 2007 > root@server.yirdis.net:/data/obj/data/src_6_2/sys/YIRDIS > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel(R) Xeon(TM) CPU 3.00GHz (3000.12-MHz 686-class CPU) > Origin = "GenuineIntel" Id = 0xf41 Stepping = 1 > > Features=0xbfebfbff > Features2=0x641d> > AMD Features=0x20000000 > Logical CPUs per core: 2 > real memory = 2147430400 (2047 MB) > avail memory = 2100547584 (2003 MB) > ACPI APIC Table: > Security auditing service present > BSM auditing present > ioapic1: Changing APIC ID to 9 > ioapic0 irqs 0-23 on motherboard > ioapic1 irqs 24-47 on motherboard > ioapic2 irqs 48-71 on motherboard > ioapic3 irqs 72-95 on motherboard > acpi0: on motherboard > acpi0: Power Button (fixed) > Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x908-0x90b on acpi0 > cpu0: on acpi0 > pcib0: on acpi0 > pci0: on pcib0 > pcib1: at device 2.0 on pci0 > pci13: on pcib1 > pcib2: at device 4.0 on pci0 > pci6: on pcib2 > pcib3: at device 0.0 on pci6 > pci7: on pcib3 > pcib4: at device 0.2 on pci6 > pci10: on pcib4 > pcib5: at device 1.0 on pci10 > pci11: on pcib5 > ste0: port 0x5000-0x507f irq 72 at > device 4.0 on pci11 > miibus0: on ste0 > ukphy0: on miibus0 > ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto > ste0: Ethernet address: 00:0d:88:fc:c8:cc > ste1: port 0x5080-0x50ff irq 73 at > device 5.0 on pci11 > miibus1: on ste1 > ukphy1: on miibus1 > ukphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto > ste1: Ethernet address: 00:0d:88:fc:c8:cd > ste2: port 0x5400-0x547f irq 74 at > device 6.0 on pci11 > miibus2: on ste2 > ukphy2: on miibus2 > ukphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto > ste2: Ethernet address: 00:0d:88:fc:c8:ce > ste3: port 0x5480-0x54ff irq 75 at > device 7.0 on pci11 > miibus3: on ste3 > ukphy3: on miibus3 > ukphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto > ste3: Ethernet address: 00:0d:88:fc:c8:cf > pcib6: at device 6.0 on pci0 > pci3: on pcib6 > pcib7: at device 28.0 on pci0 > pci2: on pcib7 > ciss0: port 0x4000-0x40ff mem > 0xfdff0000-0xfdff1fff,0xfdf80000-0xfdfbffff irq 24 at device 1.0 on pci2 > ciss0: [GIANT-LOCKED] > bge0: mem > 0xfdf70000-0xfdf7ffff irq 25 at device 2.0 on pci2 > miibus4: on bge0 > brgphy0: on miibus4 > brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, > 1000baseTX-FDX, auto > bge0: Ethernet address: 00:12:79:94:ed:12 > bge1: mem > 0xfdf60000-0xfdf6ffff irq 26 at device 2.1 on pci2 > miibus5: on bge1 > brgphy1: on miibus5 > brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, > 1000baseTX-FDX, auto > bge1: Ethernet address: 00:12:79:94:ed:11 > pci0: at device 29.0 (no driver attached) > pci0: at device 29.1 (no driver attached) > pci0: at device 29.4 (no driver attached) > pci0: at device 29.5 (no > driver attached) > pci0: at device 29.7 (no driver attached) > pcib8: at device 30.0 on pci0 > pci1: on pcib8 > pci1: at device 3.0 (no driver attached) > pci1: at device 4.0 (no driver attached) > pci1: at device 4.2 (no driver attached) > isab0: at device 31.0 on pci0 > isa0: on isab0 > atapci0: port > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x500-0x50f irq 18 at device 31.1 > on pci0 > ata0: on atapci0 > ata1: on atapci0 > acpi_tz0: on acpi0 > atkbdc0: port 0x60,0x64 irq 1 on acpi0 > atkbd0: irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > psm0: irq 12 on atkbdc0 > psm0: [GIANT-LOCKED] > psm0: model IntelliMouse Explorer, device ID 4 > sio0: port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 > sio0: type 16550A > fdc0: port 0x3f2-0x3f5 irq 6 drq 2 on acpi0 > fdc0: [FAST] > fd0: <1440-KB 3.5" drive> on fdc0 drive 0 > pmtimer0 on isa0 > orm0: at iomem > 0xc0000-0xc7fff,0xc8000-0xcbfff,0xee000-0xeffff on isa0 > sc0: at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=0x300> > sio1 at port 0x2f8-0x2ff irq 3 on isa0 > sio1: type 16550A > vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 > Timecounter "TSC" frequency 3000124702 Hz quality 800 > Timecounters tick every 1.000 msec > acd0: CDROM at ata0-master PIO4 > da0 at ciss0 bus 0 target 0 lun 0 > da0: Fixed Direct Access SCSI-0 device > da0: 135.168MB/s transfers > da0: 69459MB (142253280 512 byte sectors: 255H 32S/T 17433C) > Trying to mount root from ufs:/dev/da0s1a > Accounting enabled > [...] > > And here the netstat interface statistics: > [...] > Name Mtu Network Address Ipkts Ierrs Opkts > Oerrs Coll > ste0* 1500 00:0d:88:fc:c8:cc 0 0 0 > 0 0 > ste1* 1500 00:0d:88:fc:c8:cd 0 0 0 > 0 0 > ste2* 1500 00:0d:88:fc:c8:ce 0 0 0 > 0 0 > ste3* 1500 00:0d:88:fc:c8:cf 0 0 0 > 0 0 > bge0 1500 00:12:79:94:ed:12 11343768 2114510 20564242 > 0 0 > bge1* 1500 00:12:79:94:ed:11 0 0 0 > 0 0 > pflog 33208 0 0 0 > 0 0 > lo0 16384 60221421 0 60221421 > 0 0 > [...] > > If you have a patch for FreeBSD 6.2, i'm quite happily to test it on > our development server. > > Regards, > > Robin Gruyters > Sorry for the late reply - i was AFK for some time and didnt read mails. Patch against 6.2R attached, please let me know does it helps or not. -- Oleg. ================================================================ === Oleg Bulyzhin -- OBUL-RIPN -- OBUL-RIPE -- oleg@rinet.ru === ================================================================ --k+w/mQv8wyuph6w0 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="bge_link.r62.diff" Index: sys/dev/bge/if_bgereg.h =================================================================== RCS file: /home/ncvs/src/sys/dev/bge/if_bgereg.h,v retrieving revision 1.36.2.8.2.1 diff -u -r1.36.2.8.2.1 if_bgereg.h --- sys/dev/bge/if_bgereg.h 21 Dec 2006 21:53:54 -0000 1.36.2.8.2.1 +++ sys/dev/bge/if_bgereg.h 5 Feb 2007 23:23:22 -0000 @@ -2460,8 +2460,13 @@ uint32_t bge_tx_buf_ratio; int bge_if_flags; int bge_txcnt; - int bge_link; /* link state */ - int bge_link_evt; /* pending link event */ + uint32_t bge_sts; +#define BGE_STS_LINK 0x00000001 /* MAC link status */ +#define BGE_STS_LINK_EVT 0x00000002 /* pending link event */ +#define BGE_STS_AUTOPOLL 0x00000004 /* PHY auto-polling */ +#define BGE_STS_BIT(sc, x) ((sc)->bge_sts & (x)) +#define BGE_STS_SETBIT(sc, x) ((sc)->bge_sts |= (x)) +#define BGE_STS_CLRBIT(sc, x) ((sc)->bge_sts &= ~(x)) struct callout bge_stat_ch; char *bge_vpd_prodname; char *bge_vpd_readonly; Index: sys/dev/bge/if_bge.c =================================================================== RCS file: /home/ncvs/src/sys/dev/bge/if_bge.c,v retrieving revision 1.91.2.18.2.1 diff -u -r1.91.2.18.2.1 if_bge.c --- sys/dev/bge/if_bge.c 21 Dec 2006 21:53:54 -0000 1.91.2.18.2.1 +++ sys/dev/bge/if_bge.c 5 Feb 2007 23:23:29 -0000 @@ -602,6 +602,7 @@ /* Reading with autopolling on may trigger PCI errors */ autopoll = CSR_READ_4(sc, BGE_MI_MODE); if (autopoll & BGE_MIMODE_AUTOPOLL) { + BGE_STS_CLRBIT(sc, BGE_STS_AUTOPOLL); BGE_CLRBIT(sc, BGE_MI_MODE, BGE_MIMODE_AUTOPOLL); DELAY(40); } @@ -615,6 +616,7 @@ } if (autopoll & BGE_MIMODE_AUTOPOLL) { + BGE_STS_SETBIT(sc, BGE_STS_AUTOPOLL); BGE_SETBIT(sc, BGE_MI_MODE, BGE_MIMODE_AUTOPOLL); DELAY(40); } @@ -1456,6 +1458,7 @@ if (sc->bge_tbi) { CSR_WRITE_4(sc, BGE_MI_STS, BGE_MISTS_LINK); } else { + BGE_STS_SETBIT(sc, BGE_STS_AUTOPOLL); BGE_SETBIT(sc, BGE_MI_MODE, BGE_MIMODE_AUTOPOLL|10<<16); if (sc->bge_asicrev == BGE_ASICREV_BCM5700 && sc->bge_chipid != BGE_CHIPID_BCM5700_B2) @@ -2645,12 +2648,12 @@ /* Note link event. It will be processed by POLL_AND_CHECK_STATUS cmd */ if (statusword & BGE_STATFLAG_LINKSTATE_CHANGED) - sc->bge_link_evt++; + BGE_STS_SETBIT(sc, BGE_STS_LINK_EVT); if (cmd == POLL_AND_CHECK_STATUS) if ((sc->bge_asicrev == BGE_ASICREV_BCM5700 && sc->bge_chipid != BGE_CHIPID_BCM5700_B2) || - sc->bge_link_evt || sc->bge_tbi) + BGE_STS_BIT(sc, BGE_STS_LINK_EVT) || sc->bge_tbi) bge_link_upd(sc); sc->rxcycles = count; @@ -2699,7 +2702,7 @@ if ((sc->bge_asicrev == BGE_ASICREV_BCM5700 && sc->bge_chipid != BGE_CHIPID_BCM5700_B2) || - statusword || sc->bge_link_evt) + statusword || BGE_STS_BIT(sc, BGE_STS_LINK_EVT)) bge_link_upd(sc); if (ifp->if_drv_flags & IFF_DRV_RUNNING) { @@ -2733,8 +2736,10 @@ bge_stats_update(sc); if (!sc->bge_tbi) { - mii = device_get_softc(sc->bge_miibus); - mii_tick(mii); + if (!BGE_STS_BIT(sc, BGE_STS_LINK)) { + mii = device_get_softc(sc->bge_miibus); + mii_tick(mii); + } } else { /* * Since in TBI mode auto-polling can't be used we should poll @@ -2746,7 +2751,7 @@ if (!(sc->bge_ifp->if_capenable & IFCAP_POLLING)) #endif { - sc->bge_link_evt++; + BGE_STS_SETBIT(sc, BGE_STS_LINK_EVT); BGE_SETBIT(sc, BGE_MISC_LOCAL_CTL, BGE_MLC_INTR_SET); } } @@ -2988,7 +2993,7 @@ sc = ifp->if_softc; - if (!sc->bge_link || IFQ_DRV_IS_EMPTY(&ifp->if_snd)) + if (!BGE_STS_BIT(sc, BGE_STS_LINK) || IFQ_DRV_IS_EMPTY(&ifp->if_snd)) return; prodidx = sc->bge_tx_prodidx; @@ -3264,7 +3269,7 @@ return (0); } - sc->bge_link_evt++; + BGE_STS_SETBIT(sc, BGE_STS_LINK_EVT); mii = device_get_softc(sc->bge_miibus); if (mii->mii_instance) { struct mii_softc *miisc; @@ -3563,9 +3568,9 @@ * lead to hardware deadlock. So we just clearing MAC's link state * (PHY may still have link UP). */ - if (bootverbose && sc->bge_link) + if (bootverbose && BGE_STS_BIT(sc, BGE_STS_LINK)) if_printf(sc->bge_ifp, "link DOWN\n"); - sc->bge_link = 0; + BGE_STS_CLRBIT(sc, BGE_STS_LINK); ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE); } @@ -3623,12 +3628,12 @@ bge_link_upd(struct bge_softc *sc) { struct mii_data *mii; - uint32_t link, status; + uint32_t status; BGE_LOCK_ASSERT(sc); /* Clear 'pending link event' flag. */ - sc->bge_link_evt = 0; + BGE_STS_CLRBIT(sc, BGE_STS_LINK_EVT); /* * Process link state changes. @@ -3649,20 +3654,18 @@ sc->bge_chipid != BGE_CHIPID_BCM5700_B2) { status = CSR_READ_4(sc, BGE_MAC_STS); if (status & BGE_MACSTAT_MI_INTERRUPT) { - callout_stop(&sc->bge_stat_ch); - bge_tick_locked(sc); - mii = device_get_softc(sc->bge_miibus); - if (!sc->bge_link && + mii_pollstat(mii); + if (!BGE_STS_BIT(sc, BGE_STS_LINK) && mii->mii_media_status & IFM_ACTIVE && IFM_SUBTYPE(mii->mii_media_active) != IFM_NONE) { - sc->bge_link++; + BGE_STS_SETBIT(sc, BGE_STS_LINK); if (bootverbose) if_printf(sc->bge_ifp, "link UP\n"); - } else if (sc->bge_link && + } else if (BGE_STS_BIT(sc, BGE_STS_LINK) && (!(mii->mii_media_status & IFM_ACTIVE) || IFM_SUBTYPE(mii->mii_media_active) == IFM_NONE)) { - sc->bge_link = 0; + BGE_STS_CLRBIT(sc, BGE_STS_LINK); if (bootverbose) if_printf(sc->bge_ifp, "link DOWN\n"); } @@ -3680,8 +3683,8 @@ if (sc->bge_tbi) { status = CSR_READ_4(sc, BGE_MAC_STS); if (status & BGE_MACSTAT_TBI_PCS_SYNCHED) { - if (!sc->bge_link) { - sc->bge_link++; + if (!BGE_STS_BIT(sc, BGE_STS_LINK)) { + BGE_STS_SETBIT(sc, BGE_STS_LINK); if (sc->bge_asicrev == BGE_ASICREV_BCM5704) BGE_CLRBIT(sc, BGE_MAC_MODE, BGE_MACMODE_TBI_SEND_CFGS); @@ -3691,42 +3694,38 @@ if_link_state_change(sc->bge_ifp, LINK_STATE_UP); } - } else if (sc->bge_link) { - sc->bge_link = 0; + } else if (BGE_STS_BIT(sc, BGE_STS_LINK)) { + BGE_STS_CLRBIT(sc, BGE_STS_LINK); if (bootverbose) if_printf(sc->bge_ifp, "link DOWN\n"); if_link_state_change(sc->bge_ifp, LINK_STATE_DOWN); } - /* Discard link events for MII/GMII cards if MI auto-polling disabled */ - } else if (CSR_READ_4(sc, BGE_MI_MODE) & BGE_MIMODE_AUTOPOLL) { - /* - * Some broken BCM chips have BGE_STATFLAG_LINKSTATE_CHANGED bit - * in status word always set. Workaround this bug by reading - * PHY link status directly. - */ - link = (CSR_READ_4(sc, BGE_MI_STS) & BGE_MISTS_LINK) ? 1 : 0; - - if (link != sc->bge_link || - sc->bge_asicrev == BGE_ASICREV_BCM5700) { - callout_stop(&sc->bge_stat_ch); - bge_tick_locked(sc); - - mii = device_get_softc(sc->bge_miibus); - if (!sc->bge_link && - mii->mii_media_status & IFM_ACTIVE && - IFM_SUBTYPE(mii->mii_media_active) != IFM_NONE) { - sc->bge_link++; - if (bootverbose) - if_printf(sc->bge_ifp, "link UP\n"); - } else if (sc->bge_link && - (!(mii->mii_media_status & IFM_ACTIVE) || - IFM_SUBTYPE(mii->mii_media_active) == IFM_NONE)) { - sc->bge_link = 0; - if (bootverbose) - if_printf(sc->bge_ifp, "link DOWN\n"); - } - } - } + /* + * Discard link events for MII/GMII cards if MI auto-polling disabled. + * This should not happen since mii callouts are locked now, but + * we keep this check for debug. + */ + } else if (BGE_STS_BIT(sc, BGE_STS_AUTOPOLL)) { + mii = device_get_softc(sc->bge_miibus); + mii_pollstat(mii); + + if (!BGE_STS_BIT(sc, BGE_STS_LINK) && + mii->mii_media_status & IFM_ACTIVE && + IFM_SUBTYPE(mii->mii_media_active) != IFM_NONE) { + BGE_STS_SETBIT(sc, BGE_STS_LINK); + if (bootverbose) + if_printf(sc->bge_ifp, "link UP\n"); + } else if (BGE_STS_BIT(sc, BGE_STS_LINK) && + (!(mii->mii_media_status & IFM_ACTIVE) || + IFM_SUBTYPE(mii->mii_media_active) == IFM_NONE)) { + BGE_STS_CLRBIT(sc, BGE_STS_LINK); + if (bootverbose) + if_printf(sc->bge_ifp, "link DOWN\n"); + } + } else + /* Should not happen, see above. */ + if_printf(sc->bge_ifp, + "link event discarded: PHY auto-polling is off.\n"); /* Clear the attention. */ CSR_WRITE_4(sc, BGE_MAC_STS, BGE_MACSTAT_SYNC_CHANGED| Index: sys/dev/mii/brgphy.c =================================================================== RCS file: /home/ncvs/src/sys/dev/mii/brgphy.c,v retrieving revision 1.34.2.7 diff -u -r1.34.2.7 brgphy.c --- sys/dev/mii/brgphy.c 21 Oct 2006 00:26:41 -0000 1.34.2.7 +++ sys/dev/mii/brgphy.c 5 Feb 2007 23:23:39 -0000 @@ -464,6 +464,9 @@ return; } + if (bmsr & BRGPHY_BMSR_LINK) + sc->mii_ticks = 0; /* Reset autoneg timer. */ + switch (PHY_READ(sc, BRGPHY_MII_AUXSTS) & BRGPHY_AUXSTS_AN_RES) { case BRGPHY_RES_1000FD: --k+w/mQv8wyuph6w0--