From owner-freebsd-net@freebsd.org Tue Mar 24 21:45:36 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BD6CA26BA8E for ; Tue, 24 Mar 2020 21:45:36 +0000 (UTC) (envelope-from dan@langille.org) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48n4YD3LGbz4cjK; Tue, 24 Mar 2020 21:45:20 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 695DC5C02BD; Tue, 24 Mar 2020 17:45:12 -0400 (EDT) Received: from imap36 ([10.202.2.86]) by compute2.internal (MEProxy); Tue, 24 Mar 2020 17:45:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= mime-version:message-id:in-reply-to:references:date:from:to:cc :subject:content-type; s=fm3; bh=+6A4QNGUXgWshF8SRD55aff8GLNYeON z7poQYW7E0ek=; b=us6gpUyFxEug6y31yFOF5/olCKAC1BWhLcfNLFOpyvh/IGw BnBVf1I3Dj7vaII40ytpihfsmUB2EtOF1j8H+Nf7Gn62Tl6DW7T8DHRe72Rk2Gho nxBtVTsPsbDmsPpGd/N9OFjvHPmX+2oZcy5qP3VQ0/yRsiINHU/VuCzMn/Cv39JE xN9jjdm21AOER90CxmpzCWyEJFWfIOb3sgqusems4XILoyAUvUa82oVS4pfPcu1+ yK48ptMjfW5AwFRRYNudMwIEtFJBkMqaF8bMk8/sRXwxtEZFTHXFxsINSUK8jK6J LJhr9bTFUXltgg1fRjEB61g8EoFFt8VMRibJ3pA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=+6A4QN GUXgWshF8SRD55aff8GLNYeONz7poQYW7E0ek=; b=Hn5J6fFbBu8ourKRYnDV7u 2Hld6MxIDe8Y26pJ3194s3/inmN4yFoPJnvTA/WrNxOxkNXWeLSxJ14Zj/Z2mpcF hiYMi30t0+cSfXUDZD4oMsqClq0KVr85kjxBoLGE3fJCRR4ApDpB+JrBIApneV1g gBSFK/hIs08GVjNsdT/wc+jG8C7+QLyPyTK8LROLXswdyvv9JSrVHtWHUzCdv/gz mi6iML3kSoIRKiuZNxwJzQ0Wg518+NFO65Nw47UZa3JPjxnkQtoL6eVsEscCutyt 7HPukpwlJ7FFdLFaVDDUyjT/8B7p+g7J6F5kjfCpEaAEmnzr4Sg7AmVbTSfJjxwg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedrudehuddguddvudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefofgggkfgjfhffhffvufgtsehttdertderredtnecuhfhrohhmpedfffgr nhcunfgrnhhgihhllhgvfdcuoegurghnsehlrghnghhilhhlvgdrohhrgheqnecuffhomh grihhnpehimhhguhhrrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghm pehmrghilhhfrhhomhepuggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 3BC3A1880062; Tue, 24 Mar 2020 17:45:11 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.1.7-1021-g152deaf-fmstable-20200319v1 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <2A35EA60C3C77D438915767F458D6568B90935EC@ORSMSX111.amr.corp.intel.com> <20200322.214355.415142200.sthaug@nethelp.no> <20200322230339.GB5808@in-addr.com> <8141b371-5cdf-4f82-8959-5fa1071aa6dd@www.fastmail.com> Date: Tue, 24 Mar 2020 17:44:50 -0400 From: "Dan Langille" To: "Gary Palmer" Cc: sthaug , "Pieper, Jeffrey E" , freebsd-net@freebsd.org Subject: Re: SFP+ on PRO/10GbE Content-Type: text/plain X-Rspamd-Queue-Id: 48n4YD3LGbz4cjK X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm3 header.b=us6gpUyF; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=Hn5J6fFb; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.27 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-5.58 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm3,messagingengine.com:s=fm2]; XM_UA_NO_VERSION(0.01)[]; URIBL_BLOCKED(0.00)[imgur.com.multi.uribl.com,langille.org.multi.uribl.com,messagingengine.com.multi.uribl.com]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.27]; RCPT_COUNT_THREE(0.00)[4]; MIME_GOOD(-0.10)[text/plain]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; IP_SCORE(-3.49)[ip: (-9.84), ipnet: 66.111.4.0/24(-4.89), asn: 11403(-2.69), country: US(-0.05)]; RCVD_IN_DNSWL_LOW(-0.10)[27.4.111.66.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_WWW(0.50)[] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2020 21:45:37 -0000 On Tue, Mar 24, 2020, at 5:23 PM, Dan Langille wrote: > On Sun, Mar 22, 2020, at 7:24 PM, Dan Langille wrote: > > On Sun, Mar 22, 2020, at 7:03 PM, Gary Palmer wrote: > > > On Sun, Mar 22, 2020 at 05:11:20PM -0400, Dan Langille wrote: > > > > On Sun, Mar 22, 2020, at 4:43 PM, sthaug@nethelp.no wrote: > > > > > > Partial success. The card is now able to use an SFP+ optic. It warns me > > > > > > when the optic is installed: > > > > > > > > > > > > Mar 22 16:49:45 r720-01 kernel: WARNING: Intel (R) Network Connections are quality tested using Intel (R) Ethernet Optics. Using untested modules is not supported and may cause unstable operation or damage to the module or the adapter. Intel Corporation is not responsible for any harm caused by using untested modules. > > > > > > > > > > > > I cannot use an SFP+ optic at the switch. The connection just does not happen. > > > > > > > > > > > > If I go back to the original SFP optic, the connection occurs, as expected at 1G. > > > > > > > > > > > > On the switch side, I've tried a known good optic from an existing connection. > > > > > > > > > > > > I could install an PRO/10GbE instead, that has a built-in transceiver. I have > > > > > > two of those in use now, both working on 10G. > > > > > > > > > > Have you tried connected it to something other than the Unifi switch? > > > > > > > > The only SFP+ capable switches I have are Unifi. > > > > > > > > I just tried the other switch (US-48) which had one SFP+ port free. Same issues there. > > > > > > Did the Unifi switch see the SFP+ optics on the switch end? > > > > That port never lit up on the switch. I didn't check anything else, > > switch-related. > > I just inserted a 10G transceiver into port 8. It shows up as a 10GBase-SR > > Screen shots at https://imgur.com/a/SSEgyxN > > At the same time, on the server side, ifconfig indicates: > > status: no carrier > supported media: > media autoselect > media 10Gbase-SR > nd6 options=23 > plugged: SFP/SFP+/SFP28 10G Base-SR (LC) > vendor: AVAGO PN: AFBR-703SDZ SN: AD0946A00YB DATE: 2009-11-10 > module temperature: 36.75 C Voltage: 3.32 Volts > RX: 0.67 mW (-1.72 dBm) TX: 0.56 mW (-2.46 dBm) > > I have not yet tried moving from auto-negotiate on both ends. With the switch set to 10 Gbps FDX, this command on the server gives an error: # ifconfig ix0 media 10Gbase-SR ix0: Invalid media type! Then I tried: # ifconfig ix0 down # ifconfig ix0 up Then it worked. 10G. I moved the cable back to the original 10G transceiver where it was doing 1G. That went to 10G. Auto-negotiate is not needed. Just the up/down to reset the NIC. All good now. :) Thank you. -- Dan Langille dan@langille.org