Date: Mon, 14 Jun 2010 12:21:04 -0400 From: Alexander Sack <pisymbol@gmail.com> To: Andrew Boyer <aboyer@averesystems.com> Cc: Juli Mallett <jmallett@freebsd.org>, freebsd-net@freebsd.org, Jack Vogel <jfvogel@gmail.com> Subject: Re: Dual-rate transceivers with ixgbe? Message-ID: <AANLkTimHAH9rNU9wPgykEKwLwm4sGAYEgRx_hkdXjwGm@mail.gmail.com> In-Reply-To: <AANLkTikDM5l71isz28jJNv7nk9K-kmqTKptAC_yY3ACq@mail.gmail.com> References: <AANLkTinO9NZ8F9TeS68I2ULQgdlMGzlXkinCsywWosAM@mail.gmail.com> <AANLkTinS607kd3wc3F2WWmA6Zk9KL4GhscxEHPtcvxA5@mail.gmail.com> <AANLkTimkxOn9h6SAkTPDqfUM9kl2CZiFrZC_BuNDfRyB@mail.gmail.com> <AANLkTikcQMXk8UebmaynOeeInGiwx8yr0NMGE1yJfm8u@mail.gmail.com> <AANLkTil_YRvU54qHtIMO7mP4yYjojeHVrCHaRcl2K2Ug@mail.gmail.com> <AANLkTim5Ao9nSh6T6HF7NztLgvbTzxuVyr8lSXAJ7bMo@mail.gmail.com> <AANLkTim9-Za5mzLTw7MDAHY_TuIQsQ0SF0_1xpxyGY7v@mail.gmail.com> <AANLkTik-V2frmirwBLtg4RemdEVvPhUmVsOP7CqEkvUi@mail.gmail.com> <AANLkTikqjQic_M3mX7OTx-V0OJxbk4vzxajPmHmIUAKa@mail.gmail.com> <AANLkTikjs42mE5QHnSvZ9x9DI1xfYowvIES-DRORz6hH@mail.gmail.com> <AANLkTilqb4y0zEmom0jyg-HEi4yp8D3nkkbUWGHUjWPt@mail.gmail.com> <AANLkTinOnzvjrmmLb3w5jOYEk02CY2tWCwnKxTv8sHS8@mail.gmail.com> <82D310BD-48DE-4F79-84C6-356AB4F04C42@averesystems.com> <AANLkTikDM5l71isz28jJNv7nk9K-kmqTKptAC_yY3ACq@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jun 10, 2010 at 5:25 PM, Alexander Sack <pisymbol@gmail.com> wrote:
> On Thu, Jun 10, 2010 at 5:12 PM, Andrew Boyer <aboyer@averesystems.com> w=
rote:
>>
>> On Jun 10, 2010, at 3:59 PM, Alexander Sack wrote:
>>>
>>>> One thing that the base driver probably ought to do is not fail in
>>>> attach if there's an unrecognized SFP+ module. =A0Since we get
>>>> interrupts on module change (although this doesn't seem to always work
>>>> *entirely* right in the stock sources, mostly wrt stored values of
>>>> AUTOC and the like) it should be possible to bring the interface up
>>>> with the unsupported (and disabled) SFP+ module and do the SFP+ module
>>>> probing we already do on hot-swap.
>>>
>>> Alright, let me see if I can test that. =A0Let me rephrase so I validat=
e
>>> what you are saying:
>>>
>>> The driver can come up with an unsupported module but disable the
>>> interface (ifconfig shows the interface, etc.).
>>>
>>> If you then hot-swap a supported SFP, it should come up then with a
>>> ifconfig down/up cycle. =A0Right?
>>>
>>> As it stand now, if you load the driver with an unsupported module, it
>>> will not attach at all causing you to reload the entire driver OR
>>> reboot the box to have it reattach to the other SFP.
>>>
>>
>> We use this patch to allow the driver to attach when no module is instal=
led. =A0This might be a starting point for you. =A0I haven't tested it with=
out all of our other changes in place so my apologies if it doesn't quite w=
ork. =A0We only have Intel modules around for testing.
>>
>> -Andrew
>>
>> --- ixgbe.c =A0 =A0 2010-06-10 16:53:08.000000000 -0400
>> +++ ixgbe.c =A0 =A0 2010-06-10 16:55:26.000000000 -0400
>> @@ -566,7 +566,7 @@
>> =A0 =A0 =A0 =A0} else if (error =3D=3D IXGBE_ERR_SFP_NOT_SUPPORTED)
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0device_printf(dev,"Unsupported SFP+ Modul=
e\n");
>>
>> - =A0 =A0 =A0 if (error) {
>> + =A0 =A0 =A0 if (error && error !=3D IXGBE_ERR_SFP_NOT_PRESENT) {
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0error =3D EIO;
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0device_printf(dev,"Hardware Initializatio=
n Failure\n");
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto err_late;
>>
>> --- ixgbe_82598.c =A0 =A0 =A0 2010-06-10 16:53:24.000000000 -0400
>> +++ ixgbe_82598.c =A0 =A0 =A0 2010-06-10 16:56:31.000000000 -0400
>> @@ -257,10 +257,6 @@
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ret_val =3D ixgbe_get_sfp_init_sequence_o=
ffsets(hw,
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&list_offset,
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&data_offset);
>> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (ret_val !=3D IXGBE_SUCCESS) {
>> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ret_val =3D IXGBE_ERR_SFP_=
NOT_SUPPORTED;
>> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out;
>> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 }
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break;
>> =A0 =A0 =A0 =A0default:
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break;
>
> Thank you Andrew! =A0I am on the same path as above! (though I also had
> 82599 changes which may not be meaningful to your application).
>
> I will have access to numerous SFP/SFP+ very soon (all Finisar based -
> though I think the Intel supplied ones are Finisar too anyway). =A0 I
> will test the dual-rate as well with both 10G and 1G traffic.
>
> Stay tuned, I want to test all of this in the lab.
>
> -aps
>
A little update, I have been playing with my card over the weekend
until we had a power outage and I am without a network (don't ask how
I am writing this email right now).
Couple of tidbits/musings:
1) I don't think this card supports 1G SFPs at all at this point? It
relies on getting the right sequence offsets based on the SFP type
stored in the EEPROM which is buried in
ixgbe_identify_sfp_module_generic(). I tried using a standard 1G SFP
(Finisar) and set the sfp_type manually (I tried short range and long
range) and it doesn't work! :( (Obviously Jack, you know best!) Of
course, this maybe directly related to #3 on this list (see below).
2) I can definitely get my Finisar 10G SFP+ to work fine with my set
of patches. I see no issues with this other than policy. I will
submit very soon.
3) Like Juli, I can not get the 1G link to come up no matter how hard
I try. At this point I am about to kick start the latest Linux to
just see if it works. I stared at the driver and there were a few
minor differences, but nothing drastic. The SFP gets recognized but
the AUTOC/LINKS registers are not happy. (I do notice Jack that in
the setup sfp/link tasks, Linux sets autoneg to false explicitly
before doing the multispeed fiber stuff).
Anyway I will a more formal patch set tomorrow. I'd like to
understand why Linux works and FreeBSD does not with the 1G link in a
supported Intel dual-rate SFP+ module?
-aps
PS This card hangs my Nehalem machine but I am going to try a new bios
(v50) to see if it goes away (I am right now on a S5000PAL).
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTimHAH9rNU9wPgykEKwLwm4sGAYEgRx_hkdXjwGm>