From owner-freebsd-net@freebsd.org Thu Aug 30 22:42:37 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EEACEFB077 for ; Thu, 30 Aug 2018 22:42:37 +0000 (UTC) (envelope-from marius.h@lden.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE53D75E12; Thu, 30 Aug 2018 22:42:36 +0000 (UTC) (envelope-from marius.h@lden.org) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 3F8D021EED; Thu, 30 Aug 2018 18:42:36 -0400 (EDT) Received: from web3 ([10.202.2.213]) by compute7.internal (MEProxy); Thu, 30 Aug 2018 18:42:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lden.org; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; bh=gyr/4tsLQGqmTD317WVwHFzk/Q5ff JbB6eHdtc0ejHc=; b=DPdVF3mwxC3FdEWympiXJPEqNhoM/L1ydzWNKoVNdY6ZN 9GcBDGL2J1IIJzWC3elOoFKoFC3j1Luu1qjfdl4dYs8a5WLINXLx286Rl8CT3VWW UqGv2FSwO+NyzEN6dUBSUvSBIrbSI2LSqxvNEdjr23mVG91zEUKFdHlu0McbcBqV CVHxPcg8XttNdgDFzHgX0QyF3kqM/MgzOnCkdIR6WqzkOS3mHW/4kWJa5aRw4sjC Yirfzj/zNknxEIldzTSaxvZIsX/+AivdM4pVwxfFpKUu4umIVWfNAyK2/5glb8gN XOExtc8Z3H5sIACJDnbz1GpbWarvl11w7vZ7ktWlg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=gyr/4t sLQGqmTD317WVwHFzk/Q5ffJbB6eHdtc0ejHc=; b=qCuz7+2rUWqUcHmrddoM5v 8dbiHQ5PyJAmI1Z/QXwIcFA6lK1PzE98a9s286UNGCUCWEYOFohgajY3SFJ4Byr0 iWcDT2k5Se+u5PYC+ClvpKEyicOlvo0Es74W1B3LntJkmc4Pfj0be6TvJPeuTbMx 5Byy6MHnx0YXmXbwl9ZSEdzY/rSA4vRodsYQuq2mYxwO6sLMnMSkh/C1nPUNqIZq 4RdCJvfzZA2hUWPqVoZ96wGyHXBDUj264GlxcfbRWBJyDBof0XbmxYSLnE5l3Muv FgG6fgzzWOYTbBfKbzC6JvVXvpXqHE0hseveMPC5Nh1/v86gCmUPcZ2sZ2l7pa+g == X-ME-Proxy: X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id 2DB389E4AA; Thu, 30 Aug 2018 18:42:35 -0400 (EDT) Message-Id: <1535668955.3142238.1491912672.735C6406@webmail.messagingengine.com> From: Marius Halden To: "Rodney W. Grimes" , Navdeep Parhar Cc: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-7b72137a In-Reply-To: <201808302222.w7UMMISG028351@pdx.rh.CN85.dnsmgr.net> Subject: Re: cxl nic not working after reboot Date: Fri, 31 Aug 2018 00:42:35 +0200 References: <201808302222.w7UMMISG028351@pdx.rh.CN85.dnsmgr.net> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2018 22:42:37 -0000 On Fri, Aug 31, 2018, at 00:22, Rodney W. Grimes wrote: > > On 8/30/18 2:51 PM, Marius Halden wrote: > > > On Thu, Aug 30, 2018, at 19:27, Navdeep Parhar wrote: > > >> On 8/30/18 4:21 AM, Marius Halden wrote: > > >>> I tried to downgrade to a the previous bsdrp version we were running based on 11.1-RELEASE-p10, but it did not start working again. ifconfig output and t5nex0 devlog follows (non-working state), but it all looks fine to me (media has always been reported wrong). I did notice though that "hw_mac_init_port[0], ?" will always be logged when it works but not when when it doesn't work. I confirmed this on a box still running the old version which functions as intended. > > >>> > > >>> The ones not working has a newer firmware version than the ones working. > > >>> > > >>> Non-working: > > >>> # sysctl dev.t5nex.0.firmware_version > > >>> dev.t5nex.0.firmware_version: 1.19.1.0 > > >>> > > >>> > > >>> Working: > > >>> # sysctl dev.t5nex.0.firmware_version > > >>> dev.t5nex.0.firmware_version: 1.16.45.0 > > >>> > > >>> Any other ideas? > > >>> > > >> > > >> I'm still looking into it. The current working theory is that the peer > > >> is trying to autonegotiate when it shouldn't (it's 1Gbps optics). If > > >> you know how to disable AN on the peer this should be easy to test. > > > > > > Thanks. Unfortunately we don't have access to the peer as that's our ISPs router. According to the sysctl autonegotiate is not supported on our side. > > > > > > > I'll get a couple of 1G optics in a day or so and then I'll be able to > > try some things for myself. Wait for that or try a 1G TWINAX (copper) > > SFP+ cable in case you have one -- that should be able to link up (if > > it's really autonegotiation that's causing problems). > > I have seen some issues if one end has 1G optics, and the other > end has 1/10G optics, it wont reliably come up every time unless > you nail the other end to 1Gbps. I'm reasonably certain this isn't the issue here as I believe we use the same optics as our ISPs. In addition it reliably comes up every time with the old firmware and never with the new (after boot). -- Marius Halden