From owner-freebsd-net@freebsd.org Mon Aug 27 18:45:32 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70E7710931E2 for ; Mon, 27 Aug 2018 18:45:32 +0000 (UTC) (envelope-from marius.h@lden.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1E5587E441; Mon, 27 Aug 2018 18:45:32 +0000 (UTC) (envelope-from marius.h@lden.org) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 4A8A221A29; Mon, 27 Aug 2018 14:45:31 -0400 (EDT) Received: from web3 ([10.202.2.213]) by compute7.internal (MEProxy); Mon, 27 Aug 2018 14:45:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lden.org; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; bh=ddXzc/X6CPm0GNsLUhz5//ISisGbp /yXlNFIyX0xzy0=; b=n2qxTm/rc/JT8e7xNPAwiiTdGKbGJc2BOJ17bZgRNaevK LFZc0ujqYka11yRfUa+++UpCzYYuCjYHXlXOVgRtdMyiQg4zLPcgF6/ctZzkuicn ziuYNn05gSRskpLgCBloZ1CUq3ReuOcuiNVFAXpTv+a+q6G6rtSZDVH6LTTQoNx3 gGwatPwZ4x1tK/VeULIKEZx3gucfWUmk/gFjF2wvyCtOvDWXACfR46vNSZhanTqT U8w/tvmDeHE4vVms0IG2LUYJHT24hTIyHDL4+sIIDV4z56pS3nt+3JIovU+Oex2Q tuyAyKc95PcaM2XhKwve1UCbOY7OFcvxTmeG4t73g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=ddXzc/ X6CPm0GNsLUhz5//ISisGbp/yXlNFIyX0xzy0=; b=ddUsaAZPbz0CliIk88vql7 DmSLPriT11meePDM+ueFJpjACWrqAja9HxmbyeIWo+/ezBf3TZ00YK2T62GU/QuB WSBQlvfyEjIOzhuB94Vb7rduVP9wNlMuvS3m/y2WWVDD1JA1VI7/jSX9sei7j/MY o1io4+7mONeeKzqNEUKKn7CScLwnavdJvdy0AwqZwVwBB86a4Fmg6f7P7O6quP8p EJQC8H7ZHOGLvE8+L2gPvxnr9a/uEeJwP+OWK/8WUuIDfTNr1oVGMd1nJbkS0k44 xO87tbm80lo3XjRcfpAp9hqGvHY2Z5MSkqqc8xf2B/c8e8H/icjswqaR66pmQgHA == X-ME-Proxy: X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id CC7689E505; Mon, 27 Aug 2018 14:45:30 -0400 (EDT) Message-Id: <1535395530.234212.1487804896.64257978@webmail.messagingengine.com> From: Marius Halden To: Navdeep Parhar Cc: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-7b72137a In-Reply-To: <20180827151302.GA10167@ox> Subject: Re: cxl nic not working after reboot Date: Mon, 27 Aug 2018 20:45:30 +0200 References: <1535379569.2132387.1487489784.45227861@webmail.messagingengine.com> <20180827151302.GA10167@ox> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 18:45:32 -0000 On Mon, Aug 27, 2018, at 17:13, Navdeep Parhar wrote: > On Mon, Aug 27, 2018 at 04:19:29PM +0200, Marius Halden wrote: > > Hi, > >=20 > > We have some routers with Chelsio T540-CR NICs using 1Gbps SFPs (1000 > > Base-LX IIRC) to connect to our ISPs. After upgrading them to FreeBSD > > 11.2-p2 (BSDRP v1.91) we have run into some issues. When the machines > > are rebooted networking does not function at all through the ports > > connected with SFPs, ports connected with DAC cables work properly. > > According to ifconfig the SFPs are detected properly and they both >=20 > What exact ifconfig command do you use to bring the interfaces up? It's just the normal rc.conf lines with `ifconfig_cxl0=3D"inet =E2=80=A6"` = and `ifconfig_cxl0_ipv6=3D"inet6 =E2=80=A6"`, nothing special beyond that. > Provide the output of these commands when the link isn't up: > # ifconfig -mvvv > # sysctl -n dev.t5nex.0.misc.devlog Thanks, I'll try these when I'm back at work tomorrow and report back. > > send and receive light as they should. If we remove the SFPs from the > > port and then reinsert them in the same port everything will start > > working as it should. > >=20 > > Has anyone else observed this problem? Any suggestions for what can be > > done to solve/work around this issue without actually going to the > > datacenter would be great. >=20 > Try ifconfig down/up instead of physically removing and then reinserting > the SFP module. Does that work? ifconfig down/up did not help, I tried that first (I should have mentioned = it in the original email). Figuring out it would work by physically removin= g and reinserting the SFP was a side effect of initially trying to figure o= ut if this was a hardware issue. --=20 Marius Halden