Date: Fri, 14 Jan 2011 02:24:12 +0100 From: Marius Strobl <marius@alchemy.franken.de> To: Pyun YongHyeon <pyunyh@gmail.com> Cc: freebsd-net@freebsd.org, Lev Serebryakov <lev@serebryakov.spb.ru> Subject: Re: [patch] re(4) problems on networks with disabled autonegotiation "solver" (WAS: Juniper e3k with ports limitied to...) -- REQUEST FOR REVIEW Message-ID: <20110114012412.GK97101@alchemy.franken.de> In-Reply-To: <20110113212713.GC17502@michelle.cdnetworks.com> References: <36074996.20110112192009@serebryakov.spb.ru> <20110112213208.GD12920@michelle.cdnetworks.com> <20110112225907.GA44318@alchemy.franken.de> <20110113173925.GA49356@alchemy.franken.de> <20110113212713.GC17502@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jan 13, 2011 at 01:27:13PM -0800, Pyun YongHyeon wrote: > On Thu, Jan 13, 2011 at 06:39:25PM +0100, Marius Strobl wrote: > > On Wed, Jan 12, 2011 at 11:59:07PM +0100, Marius Strobl wrote: > > > On Wed, Jan 12, 2011 at 01:32:08PM -0800, Pyun YongHyeon wrote: > > > > On Wed, Jan 12, 2011 at 07:20:09PM +0300, Lev Serebryakov wrote: > > > > > Hello, Freebsd-net. > > > > > > > > > > Thanks to Pyun YongHyeon, who point me at fact, that rgephy(4) used > > > > > with re(4) does autonegotiation always and all other, who helps me > > > > > diagnose problem! > > > > > > > > > > I've prepared patch, which adds tunable/sysctl for rgephy(4) which > > > > > allows not to sue autonegotiation by this PHY (at user responsibility, > > > > > as here is PHYs which CAN NOT live without autonegotiation). It is OFF > > > > > by default, and in such case behavior of driver IS NOT CHANGED. > > > > > > > > > > But if it is set ON (non-zero value) before "media / mediopt" > > > > > changes via "ifconfig" autonegotiation IS NOT set with 10/100Mbit > > > > > settings. > > > > > > > > > > I've documented this new tunable in re(4) manpage, as here is no > > > > > rgephy(4) manpage. > > > > > > > > > > Tunable is per-device, not global one. > > > > > > > > > > Sysctl can be set after boot, but will affect only future ifconfig > > > > > calls, it doesn't change anything in PHY settings by itself. > > > > > > > > > > It allows fully manual setup on non-buggy hardware, which allows to > > > > > use Hetzner dedicated servers with FreeBSD without additional NIC or > > > > > gigabit connection. > > > > > > > > > > I've tested this patch on FreeBSD 8-STABLE on Hetzner server and it > > > > > allows me to get full-duplex 100Mbit connection and I got 11MiB/s from > > > > > local NFS with it. > > > > > > > > > > Without this patch FreeBSD is unusable on Hetzner dedicated servers > > > > > in newer DCs (DC 13 and DC 14). > > > > > > > > > > Patch is attached. I think, it worths to include it to base system, > > > > > as it allows use FreeBSD at least on one very large and popular > > > > > hosting provider without additional costs :) > > > > > > > > > > > > > Thanks for your work. After reading commit log of rgephy(4) I now > > > > refreshed my memory. The issue came from the reverse usage case. > > > > Suppose link partner announces auto-negotiation but you want to use > > > > 100baseTX/full-duplex. As you know this results in duplex mismatch > > > > and sometimes it couldn't establish a link on some RealTek PHYs. > > > > (Now I'm not entirely sure it was caused by the specific switch or > > > > rgephy(4) or both) > > > > And frequently, link partner(switch) is out of control from your > > > > domain and most switches are configured to use auto-negotiation by > > > > default. Using auto-negotiation in manual media configuration > > > > seemed to address the issue at that time. 1000baseT link always > > > > requires auto-negotiation but too many switches were broken with > > > > auto-negotiation so some switches are forced to use manual media > > > > configuration even in 1000baseT mode. Using auto-negotiation on > > > > rgephy(4) will also solve that case. > > > > > > > > So I have mixed feelings on how to handle both cases. Traditional > > > > way, which your patch does, used in manual configuration was to > > > > strictly honor specified manual media configuration even if it can > > > > break in some edge cases. Programming PHYs with traditional way > > > > shall also trigger other problems to drivers which correctly keep > > > > track of valid link state changes. Normally speed/duplex/flow-control > > > > changes require MAC reprogramming such that monitoring PHY's state > > > > change is essential to modern ethernet controllers. Forcing manual > > > > media configuration can make PHY drivers fail to report link state > > > > changes which in turn shall make ethernet controller not to work > > > > due to speed/duplex mismatches between PHY and MAC of ethernet > > > > controller. re(4) does not require MAC reprogramming but many other > > > > drivers that use regephy(4) may not work. However regphy(4) > > > > hardware I have still seem to correctly report link state change > > > > with manual link configuration. Not sure about old controllers > > > > though. > > > > > > > > I'm under the impression that rgephy(4)'s behavior seem to confuse > > > > users a lot since it unconditionally use auto-negotiation so I > > > > think it's better not to use auto-negotiation at all during manual > > > > media configuration and provides a way to use auto-negotiation in > > > > manual media configuration if administrator want to do that. > > > > > > Note that even the RealTek supplied driver always triggers an > > > auto-negotiation when manually setting the media though. However, > > > at the same time it also comes with tons of uncommented PHY fix-up > > > code which might be relevant for this or the previous issue. > > > Unfortunately, I didn't get to checking whether the MAC versions > > > in question are amongst the ones that get patched so far. > > > In any case I don't think we can easily change this (default) > > > behavior after such a relatively long time as it would break POLA > > > for an unknown number of users, even if it probably shouldn't have > > > been made the default in the first place (but again on the other > > > hand that's what RealTek themselves also chose to do). Also apart > > > from edges cases like the current issue the present behavior should > > > result in a setup that "just works" so I doubt it actually results > > > in confusion. > > > > > > > Independently of DSP fixes that might or might not fixes these > > issues I agree that there should be a way to turn off the use of > > auto-negotiation along with manual media configuration in rgephy(4) > > though. IMO there are some bugs and issues with the patch provided > > by Lev however: > > - It fails to reject enabling of PAUSE advertisement when not > > using auto-negotiation. > > - It has no effect on 1000base-T; as you pointed out some switches > > also require manual selection (without auto-negotiation) here as > > a workaround. > > - I don't like the idea of growing a tunable when the same can be > > achieved via ifmedia support. > > > > Therefore I'd like to commit the following patch (requires sources > > from head and stable branches), unless there's an objection or it > > doesn't work as expected: > > http://people.freebsd.org/~marius/rgephy.c.diff > > > > With this in place you should be able to manually set full-duplex > > with auto-negotiation turned off via: > > ifconfig re0 media 100baseTX mediaopt full-duplex,flag0 > > > > A full list of what that patch does is: > > - Allow IFM_FLAG0 to be set indicating that auto-negotiation with manual > > configuration, which is used to work around issues with certain setups > > (see r161237) by default, should not be triggered as it may in turn > > cause harm in some edge cases. > > - Even after masking the media with IFM_GMASK the result may have bits > > besides the duplex ones set so just comparing it with IFM_FDX may lead > > to false negatives. > > - Announce PAUSE support also for manually selected 1000BASE-T, but for > > all manually selected media types only in full-duplex mode. Announce > > asymmetric PAUSE support only for manually selected 1000BASE-T. > > - Simplify setting the manual configuration bits to only once after we > > have figured them all out. This also means we no longer unnecessarily > > update the hardware along the road. > > - Remove a stale comment. > > > > Looks good to me except one thing about IFM_FLAG0 handling. > It seems we don't have a way to register IFM_FLAG0 with > mii_phy_add_media() such that flag0 option is not passed to > MII_MEDIACHG handler which in turn makes rgephy4) think IFM_FLAG0 > was not specified. > Adding IFM_FLAG0 after registering available media with > mii_phy_add_mdia() seems to have no effect. Do we have to use > IFM_MAKEWIRD to handle this case? Err, no, actually that's the way the "don't care mask" of ifmedia is intended to work, i.e. when looking for a match the bits set there are ignored so it's possible to combine flags set in the "don't care mask" with all other media types and options added via IFM_MAKEWORD() when setting media. The downside is that given that f.e. in this case IFM_FLAG0 isn't added via IFM_MAKEWORD() it doesn't show up as a media option in the output of `ifconfig -m` (but ifconfig(8) probably could be made to list it somewhere). However, the problem of the previous patch was that one has to check for the flags additionally allowed via the "don't care mask" in mii_media.ifm_media rather than in mii_media.ifm_cur.ifm_media as the latter only contains the "resolved" media, i.e. the match found by ignoring the bits set in the "don't care mask". I've updated the patch at the above URL accordingly and based on my testing it now should actually work as expected. Sorry for the glitch. Marius setting
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110114012412.GK97101>