Date: Thu, 23 May 2013 10:56:04 -0700 From: Juli Mallett <jmallett@FreeBSD.org> To: Warner Losh <imp@bsdimp.com> Cc: Aleksandr Rybalko <ray@ddteam.net>, "freebsd-mips@FreeBSD.org" <freebsd-mips@freebsd.org> Subject: Re: Ubiquiti EdgeRouter Lite works multi-user with -CURRENT. Message-ID: <CACVs6=8FhTND_WtFm%2BmdL2e=DoNXqMsCXg_X8XTNcD9Om6SB7Q@mail.gmail.com> In-Reply-To: <07C411D8-C5D3-4CD4-896B-844F6514C3DF@bsdimp.com> References: <CACVs6=_UHMvo6DSyXzvXxJ0eCcSsC%2Bk3yZ42ia5TGzgHduT2zA@mail.gmail.com> <20130516111059.38543d57@wind.dino.sk> <20130516131642.adfae355aa3bf7767e9b56e5@ddteam.net> <20130516124248.33ae4e05@wind.dino.sk> <51952112.9010607@rewt.org.uk> <20130517192206.5db0533f@zeta.dino.sk> <51966CB6.2040701@rewt.org.uk> <CACVs6=-0URQ2f7UqVxRdpuGpf103KOW9CTF6FFCGaGhvg3jOMw@mail.gmail.com> <20130520110659.1d1d2165@zeta.dino.sk> <D1F45DEB-3C3C-42D1-8EDE-94B18AB32152@bsdimp.com> <20130520164001.5f7d99b8@zeta.dino.sk> <C48F8AE6-316B-4C4A-AD2B-739C698B0AAC@bsdimp.com> <20130520172508.087daf7b@zeta.dino.sk> <CACVs6=8_y5Rqo9UHnQwbEancfZOqrqEAhcu=EXGGSGLjvayKVg@mail.gmail.com> <20130523070225.4d9a3a59@zeta.dino.sk> <519DA801.2090205@rewt.org.uk> <20130523075537.37e4bcba@zeta.dino.sk> <CACVs6=-hWxgmUS8fN4_68-di=iZiB-pOeNfh-RfjFRiaOASNTQ@mail.gmail.com> <20130523134206.69ea6994@zeta.dino.sk> <07C411D8-C5D3-4CD4-896B-844F6514C3DF@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, May 23, 2013 at 10:43 AM, Warner Losh <imp@bsdimp.com> wrote:
>
> On May 23, 2013, at 5:42 AM, Milan Obuch wrote:
>
>> On Wed, 22 May 2013 22:59:36 -0700, Juli Mallett <jmallett@FreeBSD.org>
>> wrote:
>>
>>> On Wed, May 22, 2013 at 10:55 PM, Milan Obuch <freebsd-mips@dino.sk>
>>> wrote:
>>>> Yes, you are right - now I checked with octe0 connected to 100
>>>> megabit switch and it initializes correctly when booting. When I
>>>> plug gigabit card now instead, it does not work - no communication
>>>> on interface. Even if I do ifconfig octe down/ifconfig octe up, it
>>>> does not transmit/receive packets. So I think problem is phy link
>>>> speed change on live system. Reboot in this case is a big hackish
>>>> 'workaround' for now - good for tests, not yet fully for real work
>>>> (but if you know there will be no link speed change it is OK).
>>>
>>> The link state management is crappy, I concede. I kept it as it was
>>> in the Linux code because I wanted to be able to merge driver updates
>>> from Cavium, but that's not viable given how much they've changed the
>>> driver anyway. How long did you wait? It could take between 5
>>> seconds and a minute for link state to change at runtime in my
>>> experience. I looked into making this faster at one point and even
>>> had a patch, but don't know where it is or why I didn't commit it.
>>>
>>> If either of you wants to take a crack at fixing it I can explain what
>>> to instrument in the driver and what's likely to be the problem. Let
>>> me know if that might be useful. (Apologies of the latency is high on
>>> my responding.)
>>>
>>> Thanks,
>>> Juli.
>>
>> If you have any patches to test, some how-to what to look for and try
>> to change etc... I could try to work a bit on it. It seems there is not
>> much freely available resources for Cavium CPUs (I found cnusers.org is
>> of interest, it seems) so some leading will be good to have :)
>
> cnusers is Cavium's release vehicle for GPL'd code... There's lots of code available for the Cavium processor, but the docs are kinda hard to get a hold of...
And I'd add that the code is fairly complete and heavily-documented
(well-documented is a different question; it can be a little obtuse.)
Some people find code as useful or more useful than documentation, and
in that case the code Cavium puts out is useful-enough, although most
of it is already in src/sys/contrib/octeon-sdk, although I sometimes
have found looking at the U-Boot code helpful, too. I've never found
Cavium's documentation to be as useful as their code, but that's just
me.
In this case, I'd say that the most useful thing to instrument is
likely cvm_do_timer in sys/mips/cavium/octe/ethernet.c. I'd suggest
first changing:
138 if (priv->need_link_update) {
139 updated++;
140 taskqueue_enqueue(cvm_oct_link_taskq, &priv->link_task);
141 }
To be just
taskqueue_enqueue(cvm_oct_link_taskq, &priv->link_task);
If that helps, then I think there's two things that need to be done:
First, we shouldn't use slow-polling to do that anyway, we should just
call taskqueue_enqueue whenever we set need_link_update to 1 if it was
not 1 already. This is mostly a structural-cosmetic kind of thing.
Second, cvm_oct_rgmii_poll in ethernet-rgmii.c is probably wrong in
some way. Instrumenting it to print out the various register states
and see if a human can see link status changes even if it can't may be
helpful. Alternately, you may want/need to look at UBNT's patches for
the ERLite. I may have missed some change it requires (if so, please
let me know rather than just committing the changes; I've tried to
isolate and keep minimal vendor-specific changes since we support so
many vendors; not every vendor shares my concern, and a lot of them
make the code too complex, or other things we don't want.) It could
be just a matter of not using/programming/undermining RGMII in the way
the hardware wants, or it could be that we need a PHY driver or
something else.
Actually, looking at how the hardware is set up, it looks like we
should be using a PHY driver already, so it may be that none of that
is helpful (since the internal link status management is only used if
we don't have a PHY) and the real fix needs to happen in atphy.c. Or
that for this hardware we should just use the internal link status
management rather than using the PHY.
Thanks,
Juli.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CACVs6=8FhTND_WtFm%2BmdL2e=DoNXqMsCXg_X8XTNcD9Om6SB7Q>