From owner-freebsd-net@freebsd.org Mon Aug 27 15:13:13 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32EF8108D7F7 for ; Mon, 27 Aug 2018 15:13:13 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A65257519A for ; Mon, 27 Aug 2018 15:13:12 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-pg1-x52f.google.com with SMTP id d1-v6so1165321pgo.3 for ; Mon, 27 Aug 2018 08:13:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-disposition:in-reply-to:user-agent; bh=01aB+7gsD1h/JzQ6lzgeuYTwJ6V/dbmDTeeANnTiPHo=; b=HRBPbHtgVvnRNT5I+Yb8bECHq33KFu3LMGeWXCR6oK2meLAm+nYiVdBeUE9X8zei4D do/2TAMyeMQ3kXEIOBy+ZYIaTwT0JZCfqsixs6Zw6ytUMfaYihbW/JNh7CxOex8/b1K/ BT/eJVqFzy0T5EMBhVsJlsa7MpPwsXqI59Pv2wFalmc1KMfpUBCMtPMZHYwzDqGVBJ63 owdkITeB3MKAfw3VUbvxfHLG62gf11xosPJ/Fcf1L3AF7/7zQHAJsJ1YZyVHLqOG6dVZ i1NdrCtU7BIZjE+49Ioc6kEnUzW2NEt9vK6CrCyX7We/KgMcmeS0A8MR8I+dEyDQS3Uf /bSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=01aB+7gsD1h/JzQ6lzgeuYTwJ6V/dbmDTeeANnTiPHo=; b=Vu1sWalr5P0N+WBlxj6ZocagqXJraQJhMS8I2gMUq9qQWAfTBlpRcw6eTdiXZohvrc qoP9W6rluVnzia3JaBivMsxiu8NxEf0VI3tyXXBdhmx7G8EuXcOsL6iOW1ciAnjBv0rN UVSsSmSKUUah1KjUPlZFNWnzr54+U/B1Dfp5z6AXfbiiNmbxgOQcs1Pb30GlpvqxXqwT pTPldPE6OnGyAAOwvZRmTzS7F5i3Wtudje8gqKQTNZCqwt/bmZm560kZPXeiVvngephh +seWTXgEMxDd2l2x+qak/mSxudIcVDTpbqiYpvFcLqtB6/zMtZa8r0UDollwPOszymo6 Sfvw== X-Gm-Message-State: APzg51CFbT9w8uRjG2mbiQ5oZFVrvQf8V25lW+K/IzztjT0H6EG+CrcJ leHeWNX7pqGVoNaxOqoYDIM= X-Google-Smtp-Source: ANB0VdbJrYqAwhBOscwLAcoeV1fxr6lnPcIDjTtYvuUSD50Oqak3eqBc8gqSIEVtyPMJzqADhL/Jxw== X-Received: by 2002:a62:5302:: with SMTP id h2-v6mr15057002pfb.183.1535382791450; Mon, 27 Aug 2018 08:13:11 -0700 (PDT) Received: from ox ([2601:641:c000:b800:55d6:510e:316b:dc20]) by smtp.gmail.com with ESMTPSA id g5-v6sm20744501pgn.73.2018.08.27.08.13.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 27 Aug 2018 08:13:10 -0700 (PDT) Sender: Navdeep Parhar Date: Mon, 27 Aug 2018 08:13:02 -0700 From: Navdeep Parhar To: Marius Halden Cc: freebsd-net@freebsd.org Subject: Re: cxl nic not working after reboot Message-ID: <20180827151302.GA10167@ox> Mail-Followup-To: Marius Halden , freebsd-net@freebsd.org References: <1535379569.2132387.1487489784.45227861@webmail.messagingengine.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1535379569.2132387.1487489784.45227861@webmail.messagingengine.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 15:13:13 -0000 On Mon, Aug 27, 2018 at 04:19:29PM +0200, Marius Halden wrote: > Hi, > > We have some routers with Chelsio T540-CR NICs using 1Gbps SFPs (1000 > Base-LX IIRC) to connect to our ISPs. After upgrading them to FreeBSD > 11.2-p2 (BSDRP v1.91) we have run into some issues. When the machines > are rebooted networking does not function at all through the ports > connected with SFPs, ports connected with DAC cables work properly. > According to ifconfig the SFPs are detected properly and they both What exact ifconfig command do you use to bring the interfaces up? Provide the output of these commands when the link isn't up: # ifconfig -mvvv # sysctl -n dev.t5nex.0.misc.devlog > send and receive light as they should. If we remove the SFPs from the > port and then reinsert them in the same port everything will start > working as it should. > > Has anyone else observed this problem? Any suggestions for what can be > done to solve/work around this issue without actually going to the > datacenter would be great. Try ifconfig down/up instead of physically removing and then reinserting the SFP module. Does that work? Regards, Navdeep