From owner-freebsd-net@FreeBSD.ORG Fri Jan 14 01:44:15 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 283CA106566C for ; Fri, 14 Jan 2011 01:44:15 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id D79218FC15 for ; Fri, 14 Jan 2011 01:44:14 +0000 (UTC) Received: by iyb26 with SMTP id 26so2177812iyb.13 for ; Thu, 13 Jan 2011 17:44:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:date:to:cc:subject:message-id:reply-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=VHAILw6lgEAhgzpDUZlHXfDKBYAvUGt6sloB9lbsdXI=; b=ULAf9JWsXxJ3Tf+uxuD6DJ3fhMhChRhXcAj8uz26QxUpVkXDH7wcBNI9dwVcN1UX5v YVylE1JwtwPV32SM4ij0hLNwdN2tyNE/jvZvlFMMg+Yj3cfet/ocOHILNAs+LLsqYp5m dbbP9sHkyEb5RrDcWDJZBPu9TZ5KI3ZepADRc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=m3sy7wmlB2XqWeZhqLF1QYw9GAWSUJJN9QyOhYI0+Kj0BglUc+GmxKzfLhgG1HqQC/ Hk2lS8VufmdyO2zJJlT/Wd+ND5mh9sjedDCiYPmuwEchi01ZB9mqU8wrJ+jJJm+8q/5z T0J5n2Q4Ij5DJYMNtuizP+B75se/OxlB+rB9Q= Received: by 10.42.176.199 with SMTP id bf7mr191125icb.82.1294969454315; Thu, 13 Jan 2011 17:44:14 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id y8sm429514ica.2.2011.01.13.17.44.12 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 13 Jan 2011 17:44:13 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 13 Jan 2011 17:43:39 -0800 From: Pyun YongHyeon Date: Thu, 13 Jan 2011 17:43:39 -0800 To: Marius Strobl Message-ID: <20110114014339.GF17502@michelle.cdnetworks.com> References: <36074996.20110112192009@serebryakov.spb.ru> <20110112213208.GD12920@michelle.cdnetworks.com> <20110112225907.GA44318@alchemy.franken.de> <20110113173925.GA49356@alchemy.franken.de> <20110113212713.GC17502@michelle.cdnetworks.com> <20110114012412.GK97101@alchemy.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110114012412.GK97101@alchemy.franken.de> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, Lev Serebryakov Subject: Re: [patch] re(4) problems on networks with disabled autonegotiation "solver" (WAS: Juniper e3k with ports limitied to...) -- REQUEST FOR REVIEW X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Jan 2011 01:44:15 -0000 On Fri, Jan 14, 2011 at 02:24:12AM +0100, Marius Strobl wrote: > On Thu, Jan 13, 2011 at 01:27:13PM -0800, Pyun YongHyeon wrote: > > On Thu, Jan 13, 2011 at 06:39:25PM +0100, Marius Strobl wrote: > > > On Wed, Jan 12, 2011 at 11:59:07PM +0100, Marius Strobl wrote: > > > > On Wed, Jan 12, 2011 at 01:32:08PM -0800, Pyun YongHyeon wrote: > > > > > On Wed, Jan 12, 2011 at 07:20:09PM +0300, Lev Serebryakov wrote: > > > > > > Hello, Freebsd-net. > > > > > > > > > > > > Thanks to Pyun YongHyeon, who point me at fact, that rgephy(4) used > > > > > > with re(4) does autonegotiation always and all other, who helps me > > > > > > diagnose problem! > > > > > > > > > > > > I've prepared patch, which adds tunable/sysctl for rgephy(4) which > > > > > > allows not to sue autonegotiation by this PHY (at user responsibility, > > > > > > as here is PHYs which CAN NOT live without autonegotiation). It is OFF > > > > > > by default, and in such case behavior of driver IS NOT CHANGED. > > > > > > > > > > > > But if it is set ON (non-zero value) before "media / mediopt" > > > > > > changes via "ifconfig" autonegotiation IS NOT set with 10/100Mbit > > > > > > settings. > > > > > > > > > > > > I've documented this new tunable in re(4) manpage, as here is no > > > > > > rgephy(4) manpage. > > > > > > > > > > > > Tunable is per-device, not global one. > > > > > > > > > > > > Sysctl can be set after boot, but will affect only future ifconfig > > > > > > calls, it doesn't change anything in PHY settings by itself. > > > > > > > > > > > > It allows fully manual setup on non-buggy hardware, which allows to > > > > > > use Hetzner dedicated servers with FreeBSD without additional NIC or > > > > > > gigabit connection. > > > > > > > > > > > > I've tested this patch on FreeBSD 8-STABLE on Hetzner server and it > > > > > > allows me to get full-duplex 100Mbit connection and I got 11MiB/s from > > > > > > local NFS with it. > > > > > > > > > > > > Without this patch FreeBSD is unusable on Hetzner dedicated servers > > > > > > in newer DCs (DC 13 and DC 14). > > > > > > > > > > > > Patch is attached. I think, it worths to include it to base system, > > > > > > as it allows use FreeBSD at least on one very large and popular > > > > > > hosting provider without additional costs :) > > > > > > > > > > > > > > > > Thanks for your work. After reading commit log of rgephy(4) I now > > > > > refreshed my memory. The issue came from the reverse usage case. > > > > > Suppose link partner announces auto-negotiation but you want to use > > > > > 100baseTX/full-duplex. As you know this results in duplex mismatch > > > > > and sometimes it couldn't establish a link on some RealTek PHYs. > > > > > (Now I'm not entirely sure it was caused by the specific switch or > > > > > rgephy(4) or both) > > > > > And frequently, link partner(switch) is out of control from your > > > > > domain and most switches are configured to use auto-negotiation by > > > > > default. Using auto-negotiation in manual media configuration > > > > > seemed to address the issue at that time. 1000baseT link always > > > > > requires auto-negotiation but too many switches were broken with > > > > > auto-negotiation so some switches are forced to use manual media > > > > > configuration even in 1000baseT mode. Using auto-negotiation on > > > > > rgephy(4) will also solve that case. > > > > > > > > > > So I have mixed feelings on how to handle both cases. Traditional > > > > > way, which your patch does, used in manual configuration was to > > > > > strictly honor specified manual media configuration even if it can > > > > > break in some edge cases. Programming PHYs with traditional way > > > > > shall also trigger other problems to drivers which correctly keep > > > > > track of valid link state changes. Normally speed/duplex/flow-control > > > > > changes require MAC reprogramming such that monitoring PHY's state > > > > > change is essential to modern ethernet controllers. Forcing manual > > > > > media configuration can make PHY drivers fail to report link state > > > > > changes which in turn shall make ethernet controller not to work > > > > > due to speed/duplex mismatches between PHY and MAC of ethernet > > > > > controller. re(4) does not require MAC reprogramming but many other > > > > > drivers that use regephy(4) may not work. However regphy(4) > > > > > hardware I have still seem to correctly report link state change > > > > > with manual link configuration. Not sure about old controllers > > > > > though. > > > > > > > > > > I'm under the impression that rgephy(4)'s behavior seem to confuse > > > > > users a lot since it unconditionally use auto-negotiation so I > > > > > think it's better not to use auto-negotiation at all during manual > > > > > media configuration and provides a way to use auto-negotiation in > > > > > manual media configuration if administrator want to do that. > > > > > > > > Note that even the RealTek supplied driver always triggers an > > > > auto-negotiation when manually setting the media though. However, > > > > at the same time it also comes with tons of uncommented PHY fix-up > > > > code which might be relevant for this or the previous issue. > > > > Unfortunately, I didn't get to checking whether the MAC versions > > > > in question are amongst the ones that get patched so far. > > > > In any case I don't think we can easily change this (default) > > > > behavior after such a relatively long time as it would break POLA > > > > for an unknown number of users, even if it probably shouldn't have > > > > been made the default in the first place (but again on the other > > > > hand that's what RealTek themselves also chose to do). Also apart > > > > from edges cases like the current issue the present behavior should > > > > result in a setup that "just works" so I doubt it actually results > > > > in confusion. > > > > > > > > > > Independently of DSP fixes that might or might not fixes these > > > issues I agree that there should be a way to turn off the use of > > > auto-negotiation along with manual media configuration in rgephy(4) > > > though. IMO there are some bugs and issues with the patch provided > > > by Lev however: > > > - It fails to reject enabling of PAUSE advertisement when not > > > using auto-negotiation. > > > - It has no effect on 1000base-T; as you pointed out some switches > > > also require manual selection (without auto-negotiation) here as > > > a workaround. > > > - I don't like the idea of growing a tunable when the same can be > > > achieved via ifmedia support. > > > > > > Therefore I'd like to commit the following patch (requires sources > > > from head and stable branches), unless there's an objection or it > > > doesn't work as expected: > > > http://people.freebsd.org/~marius/rgephy.c.diff > > > > > > With this in place you should be able to manually set full-duplex > > > with auto-negotiation turned off via: > > > ifconfig re0 media 100baseTX mediaopt full-duplex,flag0 > > > > > > A full list of what that patch does is: > > > - Allow IFM_FLAG0 to be set indicating that auto-negotiation with manual > > > configuration, which is used to work around issues with certain setups > > > (see r161237) by default, should not be triggered as it may in turn > > > cause harm in some edge cases. > > > - Even after masking the media with IFM_GMASK the result may have bits > > > besides the duplex ones set so just comparing it with IFM_FDX may lead > > > to false negatives. > > > - Announce PAUSE support also for manually selected 1000BASE-T, but for > > > all manually selected media types only in full-duplex mode. Announce > > > asymmetric PAUSE support only for manually selected 1000BASE-T. > > > - Simplify setting the manual configuration bits to only once after we > > > have figured them all out. This also means we no longer unnecessarily > > > update the hardware along the road. > > > - Remove a stale comment. > > > > > > > Looks good to me except one thing about IFM_FLAG0 handling. > > It seems we don't have a way to register IFM_FLAG0 with > > mii_phy_add_media() such that flag0 option is not passed to > > MII_MEDIACHG handler which in turn makes rgephy4) think IFM_FLAG0 > > was not specified. > > Adding IFM_FLAG0 after registering available media with > > mii_phy_add_mdia() seems to have no effect. Do we have to use > > IFM_MAKEWIRD to handle this case? > > Err, no, actually that's the way the "don't care mask" of ifmedia > is intended to work, i.e. when looking for a match the bits set > there are ignored so it's possible to combine flags set in the > "don't care mask" with all other media types and options added > via IFM_MAKEWORD() when setting media. The downside is that given > that f.e. in this case IFM_FLAG0 isn't added via IFM_MAKEWORD() it > doesn't show up as a media option in the output of `ifconfig -m` > (but ifconfig(8) probably could be made to list it somewhere). > However, the problem of the previous patch was that one has to > check for the flags additionally allowed via the "don't care mask" > in mii_media.ifm_media rather than in mii_media.ifm_cur.ifm_media > as the latter only contains the "resolved" media, i.e. the match > found by ignoring the bits set in the "don't care mask". I've Hmm, that's confusing but I see your point. :-) > updated the patch at the above URL accordingly and based on my > testing it now should actually work as expected. Sorry for the > glitch. I can confirm your latest patch works as expected. Thanks for the patch! > > Marius > > setting