Date: Thu, 23 May 2013 19:16:29 +0100 From: Joe Holden <lists@rewt.org.uk> To: Juli Mallett <jmallett@FreeBSD.org> Cc: Aleksandr Rybalko <ray@ddteam.net>, "freebsd-mips@FreeBSD.org" <freebsd-mips@freebsd.org> Subject: Re: Ubiquiti EdgeRouter Lite works multi-user with -CURRENT. Message-ID: <519E5CFD.7060008@rewt.org.uk> In-Reply-To: <CACVs6=8FhTND_WtFm%2BmdL2e=DoNXqMsCXg_X8XTNcD9Om6SB7Q@mail.gmail.com> References: <CACVs6=_UHMvo6DSyXzvXxJ0eCcSsC%2Bk3yZ42ia5TGzgHduT2zA@mail.gmail.com> <20130516131642.adfae355aa3bf7767e9b56e5@ddteam.net> <20130516124248.33ae4e05@wind.dino.sk> <51952112.9010607@rewt.org.uk> <20130517192206.5db0533f@zeta.dino.sk> <51966CB6.2040701@rewt.org.uk> <CACVs6=-0URQ2f7UqVxRdpuGpf103KOW9CTF6FFCGaGhvg3jOMw@mail.gmail.com> <20130520110659.1d1d2165@zeta.dino.sk> <D1F45DEB-3C3C-42D1-8EDE-94B18AB32152@bsdimp.com> <20130520164001.5f7d99b8@zeta.dino.sk> <C48F8AE6-316B-4C4A-AD2B-739C698B0AAC@bsdimp.com> <20130520172508.087daf7b@zeta.dino.sk> <CACVs6=8_y5Rqo9UHnQwbEancfZOqrqEAhcu=EXGGSGLjvayKVg@mail.gmail.com> <20130523070225.4d9a3a59@zeta.dino.sk> <519DA801.2090205@rewt.org.uk> <20130523075537.37e4bcba@zeta.dino.sk> <CACVs6=-hWxgmUS8fN4_68-di=iZiB-pOeNfh-RfjFRiaOASNTQ@mail.gmail.com> <20130523134206.69ea6994@zeta.dino.sk> <07C411D8-C5D3-4CD4-896B-844F6514C3DF@bsdimp.com> <CACVs6=8FhTND_WtFm%2BmdL2e=DoNXqMsCXg_X8XTNcD9Om6SB7Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Juli Mallett wrote:
> On Thu, May 23, 2013 at 10:43 AM, Warner Losh <imp@bsdimp.com> wrote:
>> On May 23, 2013, at 5:42 AM, Milan Obuch wrote:
>>
>>> On Wed, 22 May 2013 22:59:36 -0700, Juli Mallett <jmallett@FreeBSD.org>
>>> wrote:
>>>
>>>> On Wed, May 22, 2013 at 10:55 PM, Milan Obuch <freebsd-mips@dino.sk>
>>>> wrote:
>>>>> Yes, you are right - now I checked with octe0 connected to 100
>>>>> megabit switch and it initializes correctly when booting. When I
>>>>> plug gigabit card now instead, it does not work - no communication
>>>>> on interface. Even if I do ifconfig octe down/ifconfig octe up, it
>>>>> does not transmit/receive packets. So I think problem is phy link
>>>>> speed change on live system. Reboot in this case is a big hackish
>>>>> 'workaround' for now - good for tests, not yet fully for real work
>>>>> (but if you know there will be no link speed change it is OK).
>>>> The link state management is crappy, I concede. I kept it as it was
>>>> in the Linux code because I wanted to be able to merge driver updates
>>>> from Cavium, but that's not viable given how much they've changed the
>>>> driver anyway. How long did you wait? It could take between 5
>>>> seconds and a minute for link state to change at runtime in my
>>>> experience. I looked into making this faster at one point and even
>>>> had a patch, but don't know where it is or why I didn't commit it.
>>>>
>>>> If either of you wants to take a crack at fixing it I can explain what
>>>> to instrument in the driver and what's likely to be the problem. Let
>>>> me know if that might be useful. (Apologies of the latency is high on
>>>> my responding.)
>>>>
>>>> Thanks,
>>>> Juli.
>>> If you have any patches to test, some how-to what to look for and try
>>> to change etc... I could try to work a bit on it. It seems there is not
>>> much freely available resources for Cavium CPUs (I found cnusers.org is
>>> of interest, it seems) so some leading will be good to have :)
>> cnusers is Cavium's release vehicle for GPL'd code... There's lots of code available for the Cavium processor, but the docs are kinda hard to get a hold of...
>
> And I'd add that the code is fairly complete and heavily-documented
> (well-documented is a different question; it can be a little obtuse.)
> Some people find code as useful or more useful than documentation, and
> in that case the code Cavium puts out is useful-enough, although most
> of it is already in src/sys/contrib/octeon-sdk, although I sometimes
> have found looking at the U-Boot code helpful, too. I've never found
> Cavium's documentation to be as useful as their code, but that's just
> me.
>
> In this case, I'd say that the most useful thing to instrument is
> likely cvm_do_timer in sys/mips/cavium/octe/ethernet.c. I'd suggest
> first changing:
>
> 138 if (priv->need_link_update) {
> 139 updated++;
> 140 taskqueue_enqueue(cvm_oct_link_taskq, &priv->link_task);
> 141 }
>
> To be just
>
> taskqueue_enqueue(cvm_oct_link_taskq, &priv->link_task);
>
> If that helps, then I think there's two things that need to be done:
>
> First, we shouldn't use slow-polling to do that anyway, we should just
> call taskqueue_enqueue whenever we set need_link_update to 1 if it was
> not 1 already. This is mostly a structural-cosmetic kind of thing.
>
> Second, cvm_oct_rgmii_poll in ethernet-rgmii.c is probably wrong in
> some way. Instrumenting it to print out the various register states
> and see if a human can see link status changes even if it can't may be
> helpful. Alternately, you may want/need to look at UBNT's patches for
> the ERLite. I may have missed some change it requires (if so, please
> let me know rather than just committing the changes; I've tried to
> isolate and keep minimal vendor-specific changes since we support so
> many vendors; not every vendor shares my concern, and a lot of them
> make the code too complex, or other things we don't want.) It could
> be just a matter of not using/programming/undermining RGMII in the way
> the hardware wants, or it could be that we need a PHY driver or
> something else.
>
> Actually, looking at how the hardware is set up, it looks like we
> should be using a PHY driver already, so it may be that none of that
> is helpful (since the internal link status management is only used if
> we don't have a PHY) and the real fix needs to happen in atphy.c. Or
> that for this hardware we should just use the internal link status
> management rather than using the PHY.
>
> Thanks,
> Juli.
Yeah it doesn't appear to just be a delay as the link still hasn't
started working since this morning - I did note that the atheros phy
stuff in the UBNT release identifies itself as a modified ubnt one, so
there may well be some slight change that needs doing - well outside of
the scope of my skills though :(
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?519E5CFD.7060008>