Date: Sat, 25 May 2013 10:32:54 -0700 From: Juli Mallett <jmallett@FreeBSD.org> To: Milan Obuch <freebsd-mips@dino.sk> Cc: Aleksandr Rybalko <ray@ddteam.net>, "freebsd-mips@FreeBSD.org" <freebsd-mips@freebsd.org> Subject: Re: Ubiquiti EdgeRouter Lite works multi-user with -CURRENT. Message-ID: <CACVs6=8LFHETGzQ1gcpOV-xnyDQ1acV2nhn6-5BE9d1TW6MGMg@mail.gmail.com> In-Reply-To: <20130525163545.1fb37ea5@zeta.dino.sk> References: <CACVs6=_UHMvo6DSyXzvXxJ0eCcSsC%2Bk3yZ42ia5TGzgHduT2zA@mail.gmail.com> <20130516131642.adfae355aa3bf7767e9b56e5@ddteam.net> <20130516124248.33ae4e05@wind.dino.sk> <51952112.9010607@rewt.org.uk> <20130517192206.5db0533f@zeta.dino.sk> <51966CB6.2040701@rewt.org.uk> <CACVs6=-0URQ2f7UqVxRdpuGpf103KOW9CTF6FFCGaGhvg3jOMw@mail.gmail.com> <20130520110659.1d1d2165@zeta.dino.sk> <D1F45DEB-3C3C-42D1-8EDE-94B18AB32152@bsdimp.com> <20130520164001.5f7d99b8@zeta.dino.sk> <C48F8AE6-316B-4C4A-AD2B-739C698B0AAC@bsdimp.com> <20130520172508.087daf7b@zeta.dino.sk> <CACVs6=8_y5Rqo9UHnQwbEancfZOqrqEAhcu=EXGGSGLjvayKVg@mail.gmail.com> <20130523070225.4d9a3a59@zeta.dino.sk> <519DA801.2090205@rewt.org.uk> <20130523075537.37e4bcba@zeta.dino.sk> <CACVs6=-hWxgmUS8fN4_68-di=iZiB-pOeNfh-RfjFRiaOASNTQ@mail.gmail.com> <20130523134206.69ea6994@zeta.dino.sk> <07C411D8-C5D3-4CD4-896B-844F6514C3DF@bsdimp.com> <CACVs6=8FhTND_WtFm%2BmdL2e=DoNXqMsCXg_X8XTNcD9Om6SB7Q@mail.gmail.com> <20130525163545.1fb37ea5@zeta.dino.sk>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, May 25, 2013 at 7:35 AM, Milan Obuch <freebsd-mips@dino.sk> wrote:
> On Thu, 23 May 2013 10:56:04 -0700, Juli Mallett <jmallett@FreeBSD.org>
> wrote:
>
> [ snip ]
>
> > In this case, I'd say that the most useful thing to instrument is
> > likely cvm_do_timer in sys/mips/cavium/octe/ethernet.c. I'd suggest
> > first changing:
> >
> > 138 if (priv->need_link_update) {
> > 139 updated++;
> > 140 taskqueue_enqueue(cvm_oct_link_taskq, &priv->link_task);
> > 141 }
> >
> > To be just
> >
> > taskqueue_enqueue(cvm_oct_link_taskq, &priv->link_task);
> >
>
> It took me some time to respond for some reasons, anyway I built a
> kernel with this change and there is something clearly wrong. There is
> basically no change in behavior, if booted with gigabit connection, it
> works with gigabit speed but not with 100 megabit and vice versa. Just
> some cometic change - once per two seconds or so I keep getting on
> console following
>
> octe0: Link down
> octe1: Link down
> octe2: Link down
>
> It looks like some phy register is not read correctly or not correct
> phy register is read and acted upon. Just a guess, of course.
> Interesting thing also not yet mentioned here - ifconfig octeN output
> show correct speed in its media: line.
>
> Also, when changing cables, I got following printed on console:
>
> Port 0 receive error code 13, packet dropped
>
> No idea it if shows something, for now.
>
> > If that helps, then I think there's two things that need to be done:
> >
> > First, we shouldn't use slow-polling to do that anyway, we should just
> > call taskqueue_enqueue whenever we set need_link_update to 1 if it was
> > not 1 already. This is mostly a structural-cosmetic kind of thing.
> >
> > Second, cvm_oct_rgmii_poll in ethernet-rgmii.c is probably wrong in
> > some way. Instrumenting it to print out the various register states
> > and see if a human can see link status changes even if it can't may be
> > helpful. Alternately, you may want/need to look at UBNT's patches for
> > the ERLite. I may have missed some change it requires (if so, please
> > let me know rather than just committing the changes; I've tried to
> > isolate and keep minimal vendor-specific changes since we support so
> > many vendors; not every vendor shares my concern, and a lot of them
> > make the code too complex, or other things we don't want.) It could
> > be just a matter of not using/programming/undermining RGMII in the way
> > the hardware wants, or it could be that we need a PHY driver or
> > something else.
> >
> > Actually, looking at how the hardware is set up, it looks like we
> > should be using a PHY driver already, so it may be that none of that
> > is helpful (since the internal link status management is only used if
> > we don't have a PHY) and the real fix needs to happen in atphy.c. Or
> > that for this hardware we should just use the internal link status
> > management rather than using the PHY.
> >
>
> In dmesg, following is relevant to ethernet, I think:
>
> octebus0: <Cavium Octeon Ethernet pseudo-bus> on ciu0
> Interface 0 has 3 ports (RGMII)
> Warning: Enabling IPD when IPD already enabled.
> Warning: Enabling PKO when PKO already enabled.
> octe0: <Cavium Octeon RGMII Ethernet> on octebus0
> miibus0: <MII bus> on octe0
> atphy0: <Atheros F1 10/100/1000 PHY> PHY 7 on miibus0
> atphy0: OUI 0x00c82e, model 0x0007, rev. 2
> atphy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX,
> 1000baseSX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto octe0: bpf
> attached octe0: Ethernet address: dc:9f:db:29:a3:92
> octe1: <Cavium Octeon RGMII Ethernet> on octebus0
> miibus1: <MII bus> on octe1
> atphy1: <Atheros F1 10/100/1000 PHY> PHY 6 on miibus1
> atphy1: OUI 0x00c82e, model 0x0007, rev. 2
> atphy1: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX,
> 1000baseSX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto octe1: bpf
> attached octe1: Ethernet address: dc:9f:db:29:a3:93
> octe2: <Cavium Octeon RGMII Ethernet> on octebus0
> miibus2: <MII bus> on octe2
> atphy2: <Atheros F1 10/100/1000 PHY> PHY 5 on miibus2
> atphy2: OUI 0x00c82e, model 0x0007, rev. 2
> atphy2: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX,
> 1000baseSX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto octe2: bpf
> attached octe2: Ethernet address: dc:9f:db:29:a3:94
>
> so maybe something should be done in atphy driver as someone mentioned
> later in this thread or between phy driver and some cavium code... I am
> going to try to undestand the code in cavium/octe directory, but any
> hint would be welcome for sure.
>
Ok, well, I have a slightly-radical hint :)
In cvm_oct_mdio_setup_device, set phy_id to -1 instead of to
cvmx_helper_board_get_mii_address(...). It's a silly thought, but I think
there's a chance it might work. This would be using the internal link
state management rather than using a PHY. (This isn't the right change to
make to effect that, but it's worth a shot.)
Thanks,
Juli.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CACVs6=8LFHETGzQ1gcpOV-xnyDQ1acV2nhn6-5BE9d1TW6MGMg>