From owner-freebsd-net@FreeBSD.ORG Mon Jun 14 16:21:06 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7FDAE106566B; Mon, 14 Jun 2010 16:21:06 +0000 (UTC) (envelope-from pisymbol@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0E4DD8FC14; Mon, 14 Jun 2010 16:21:05 +0000 (UTC) Received: by gyh20 with SMTP id 20so3253199gyh.13 for ; Mon, 14 Jun 2010 09:21:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=bslV8QVzBRh2wFtLk7n7Av1wMc9vo/EHSTEHPtgYDJk=; b=Tr5TkxI+SN5rb8pTwPHs91n+NZcAAi7cYqYted1aa058cNhyKQ4hVp7yvmvpHrDK4x zh23gXG9Px6jsCg3OiU9IAJL26JEMnuSjC1QnoCPXL0AHuuECuiySecUaulsuulxHSa5 /fI86E+WW4d1fBmCZXut8NJkXEFI3xRIDtOl4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=ZJDK75BAupXFWMjdMxj2ez5PDDEAOMueNh3oKjQvnCT2uOpkLj1lja1OlpsOjsh22u jMcvCBM6fwo5fpYxAypmlOPKBN1ATDgZSNjGRbDJvv/81mb/7L21XTxNNy3NMZurZ5mY oA760oOrrJHbQgiLwEDZfed8rIVnLjVioJrFA= MIME-Version: 1.0 Received: by 10.101.134.28 with SMTP id l28mr4987852ann.5.1276532465021; Mon, 14 Jun 2010 09:21:05 -0700 (PDT) Received: by 10.100.137.4 with HTTP; Mon, 14 Jun 2010 09:21:04 -0700 (PDT) In-Reply-To: References: <82D310BD-48DE-4F79-84C6-356AB4F04C42@averesystems.com> Date: Mon, 14 Jun 2010 12:21:04 -0400 Message-ID: From: Alexander Sack To: Andrew Boyer Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Juli Mallett , freebsd-net@freebsd.org, Jack Vogel Subject: Re: Dual-rate transceivers with ixgbe? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jun 2010 16:21:06 -0000 On Thu, Jun 10, 2010 at 5:25 PM, Alexander Sack wrote: > On Thu, Jun 10, 2010 at 5:12 PM, Andrew Boyer w= rote: >> >> On Jun 10, 2010, at 3:59 PM, Alexander Sack wrote: >>> >>>> One thing that the base driver probably ought to do is not fail in >>>> attach if there's an unrecognized SFP+ module. =A0Since we get >>>> interrupts on module change (although this doesn't seem to always work >>>> *entirely* right in the stock sources, mostly wrt stored values of >>>> AUTOC and the like) it should be possible to bring the interface up >>>> with the unsupported (and disabled) SFP+ module and do the SFP+ module >>>> probing we already do on hot-swap. >>> >>> Alright, let me see if I can test that. =A0Let me rephrase so I validat= e >>> what you are saying: >>> >>> The driver can come up with an unsupported module but disable the >>> interface (ifconfig shows the interface, etc.). >>> >>> If you then hot-swap a supported SFP, it should come up then with a >>> ifconfig down/up cycle. =A0Right? >>> >>> As it stand now, if you load the driver with an unsupported module, it >>> will not attach at all causing you to reload the entire driver OR >>> reboot the box to have it reattach to the other SFP. >>> >> >> We use this patch to allow the driver to attach when no module is instal= led. =A0This might be a starting point for you. =A0I haven't tested it with= out all of our other changes in place so my apologies if it doesn't quite w= ork. =A0We only have Intel modules around for testing. >> >> -Andrew >> >> --- ixgbe.c =A0 =A0 2010-06-10 16:53:08.000000000 -0400 >> +++ ixgbe.c =A0 =A0 2010-06-10 16:55:26.000000000 -0400 >> @@ -566,7 +566,7 @@ >> =A0 =A0 =A0 =A0} else if (error =3D=3D IXGBE_ERR_SFP_NOT_SUPPORTED) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0device_printf(dev,"Unsupported SFP+ Modul= e\n"); >> >> - =A0 =A0 =A0 if (error) { >> + =A0 =A0 =A0 if (error && error !=3D IXGBE_ERR_SFP_NOT_PRESENT) { >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0error =3D EIO; >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0device_printf(dev,"Hardware Initializatio= n Failure\n"); >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto err_late; >> >> --- ixgbe_82598.c =A0 =A0 =A0 2010-06-10 16:53:24.000000000 -0400 >> +++ ixgbe_82598.c =A0 =A0 =A0 2010-06-10 16:56:31.000000000 -0400 >> @@ -257,10 +257,6 @@ >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ret_val =3D ixgbe_get_sfp_init_sequence_o= ffsets(hw, >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&list_offset, >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&data_offset); >> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (ret_val !=3D IXGBE_SUCCESS) { >> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ret_val =3D IXGBE_ERR_SFP_= NOT_SUPPORTED; >> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out; >> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 } >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; >> =A0 =A0 =A0 =A0default: >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > > Thank you Andrew! =A0I am on the same path as above! (though I also had > 82599 changes which may not be meaningful to your application). > > I will have access to numerous SFP/SFP+ very soon (all Finisar based - > though I think the Intel supplied ones are Finisar too anyway). =A0 I > will test the dual-rate as well with both 10G and 1G traffic. > > Stay tuned, I want to test all of this in the lab. > > -aps > A little update, I have been playing with my card over the weekend until we had a power outage and I am without a network (don't ask how I am writing this email right now). Couple of tidbits/musings: 1) I don't think this card supports 1G SFPs at all at this point? It relies on getting the right sequence offsets based on the SFP type stored in the EEPROM which is buried in ixgbe_identify_sfp_module_generic(). I tried using a standard 1G SFP (Finisar) and set the sfp_type manually (I tried short range and long range) and it doesn't work! :( (Obviously Jack, you know best!) Of course, this maybe directly related to #3 on this list (see below). 2) I can definitely get my Finisar 10G SFP+ to work fine with my set of patches. I see no issues with this other than policy. I will submit very soon. 3) Like Juli, I can not get the 1G link to come up no matter how hard I try. At this point I am about to kick start the latest Linux to just see if it works. I stared at the driver and there were a few minor differences, but nothing drastic. The SFP gets recognized but the AUTOC/LINKS registers are not happy. (I do notice Jack that in the setup sfp/link tasks, Linux sets autoneg to false explicitly before doing the multispeed fiber stuff). Anyway I will a more formal patch set tomorrow. I'd like to understand why Linux works and FreeBSD does not with the 1G link in a supported Intel dual-rate SFP+ module? -aps PS This card hangs my Nehalem machine but I am going to try a new bios (v50) to see if it goes away (I am right now on a S5000PAL).