From owner-freebsd-net@freebsd.org Thu Aug 23 07:51:25 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 850371084D52 for ; Thu, 23 Aug 2018 07:51:25 +0000 (UTC) (envelope-from bogorodskiy@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 1545E85EB9 for ; Thu, 23 Aug 2018 07:51:25 +0000 (UTC) (envelope-from bogorodskiy@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id CE3FC1084D51; Thu, 23 Aug 2018 07:51:24 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8381E1084D50 for ; Thu, 23 Aug 2018 07:51:24 +0000 (UTC) (envelope-from bogorodskiy@gmail.com) Received: from mail-ed1-x541.google.com (mail-ed1-x541.google.com [IPv6:2a00:1450:4864:20::541]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EEB1F85EB2 for ; Thu, 23 Aug 2018 07:51:23 +0000 (UTC) (envelope-from bogorodskiy@gmail.com) Received: by mail-ed1-x541.google.com with SMTP id b10-v6so2998762eds.4 for ; Thu, 23 Aug 2018 00:51:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=G5eptRhaaC9KGtAeTrlqgUCI9mFTM1C1jz7o6XlgWxI=; b=IG/Il4WceoCvKSxR8XzOE2h3hlCHF4H/1hQyxOvZpl73FDeC8FvZVXIJ6NaaUFHeAM VDAq9i0An7HD0TA75uXk7swGyLBu0ks2/Q1Dzi8acZ180mOxhZ1QJ7KAI+HWvtXqFJ0Q mdCfLK8PmyPHxLmBwV4hai7DVZrfS0wexGg2b9XDp5pEU1ikW/lI4oISW78k6TqEwSYp Rarbb9xq6Dj1qghIIksIT0O0bWy5j99zUWPKI3MuH/MPp2+kRWc03s+pUV5C79DJqnyP MbpZh+aLFo211ZRb1M9R+xjLw1Xm9Y5rR/xGC6fDMjvLUlnaF3ifKBVUZ+Jvc0CBh3Nm qI6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=G5eptRhaaC9KGtAeTrlqgUCI9mFTM1C1jz7o6XlgWxI=; b=TI8/V29RoE38dmIkwi5SgapbrRqclw2VlRs28JmIlpRUdNC05EgyXYF10jlv23YdzA MeLoUEn3udBlxrjlDhZMY7MjzKTtqi6SieirjdZ+s+H/afQLj32fZLCnWrNGcYNMOnLx LT3rgY+/LTEeLj1+ZK5BjX0zq/9LeSCEjX+ITfL0ar5EUYYMyvkGEpX34wbNIBOlkERx K6sMgFaSJG9ki0wtOd1o2Adg5cVhtOtSybk28AaaaLGfLw/Bdtmw71HL5R8iI1i0rMax GwDOJrkxN9RnfyEcqG5iSx2NpCI+ZJLSOcWGDRlwIi3dRZq3ZJ7wXwlhlG4OgG5cB8J/ wW2A== X-Gm-Message-State: AOUpUlEY1eojyn/UmABtXyTGvZLGB4+y79tq5lVL7FOz+aebaIPWvm1d Pgu5jvJRs3aTe5PLhmpSmWwYsyr8 X-Google-Smtp-Source: AA+uWPzt9pBAjKkXzAQjk5FKUyPn3ASMwoggOFrabOdyoETxsMpZUtAO7onnuRKo+Eg57vWDNpmxqA== X-Received: by 2002:a50:e146:: with SMTP id i6-v6mr71034536edl.204.1535010682956; Thu, 23 Aug 2018 00:51:22 -0700 (PDT) Received: from kloomba ([77.94.198.250]) by smtp.gmail.com with ESMTPSA id i3-v6sm2627339eda.84.2018.08.23.00.51.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Aug 2018 00:51:22 -0700 (PDT) Date: Thu, 23 Aug 2018 11:51:18 +0400 From: Roman Bogorodskiy To: Kevin Bowling Cc: net@freebsd.org Subject: Re: igb: device_attach: igb1 attach returned 6 Message-ID: <20180823075117.GB4446@kloomba> References: <20180819142106.GA1539@kloomba> <20180822090811.GA1691@kloomba> <20180823071626.GA4446@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="s2ZSL+KKDSLx8OML" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2018 07:51:25 -0000 --s2ZSL+KKDSLx8OML Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Kevin Bowling wrote: > Well I took a look at the phy code and actually don't see that we > abosrbed those commits. I'll try and audit e1000 vs linux in the next > 24 hours but in the mean time if you could confirm linux-next or the > like fix or don't fix the phy we can at least know if this is the > right path or if we need to ask Intel networking for help. Yeah, I see the code that resets PHY in e1000_82575.c, e1000_setup_copper_link_82575(), but I don't see this being called at all in my dmesg output, so I guess removing that code will not change anything. I wasn't able to find a code similar to igb_write_phy_reg_82580(hw, I347AT4_PAGE_SELECT, 0); from that net-next patch. There's a code in e1000_phy.c e1000_get_cable_length_m88_gen2() that writes to I347AT4_PAGE_SELECT (it uses 0x07 or 0x05 instead of 0 though), b= ut I don't see it in dmesg either... I'll try to find a spare drive to install Linux, but I'm not sure how soon I'll be able to do that... > Regards, >=20 > On Thu, Aug 23, 2018 at 12:19 AM, Kevin Bowling > wrote: > > Great thanks for the info about it not working on Ubuntu too, that > > narrows down the problem space considerably. > > > > Can you try that patch on FreeBSD -CURRENT? It should apply with > > minimal changes the phy code is shared by multiple OSes. If it works > > we can commit. > > > > Regards, > > > > On Thu, Aug 23, 2018 at 12:16 AM, Roman Bogorodskiy > > wrote: > >> I haven't used this hardware before, so I don't if it ever worked. I > >> only tried 11.2 and it doesn't work. I've also tried Ubuntu 18.04 and = it > >> doesn't work as well. Ubuntu prints the following error: > >> > >> [ 343.053031] igb: Intel(R) Gigabit Ethernet Network Driver - version= 5.4.0-k > >> [ 343.053032] igb: Copyright (c) 2007-2014 Intel Corporation. > >> [ 343.415346] igb 0000:01:00.0: added PHC on eth0 > >> [ 343.415349] igb 0000:01:00.0: Intel(R) Gigabit Ethernet Network Con= nection > >> [ 343.415352] igb 0000:01:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 00:12:c= 0:00:e9:14 > >> [ 343.415427] igb 0000:01:00.0: eth0: PBA No: 106100-000 > >> [ 343.415430] igb 0000:01:00.0: Using MSI-X interrupts. 4 rx queue(s)= , 4 tx queue(s) > >> [ 343.416422] igb 0000:01:00.0 enp1s0f0: renamed from eth0 > >> [ 343.416827] igb: probe of 0000:01:00.1 failed with error -3 > >> [ 343.485242] IPv6: ADDRCONF(NETDEV_UP): enp1s0f0: link is not ready > >> [ 343.777894] igb 0000:01:00.2: added PHC on eth0 > >> [ 343.777897] igb 0000:01:00.2: Intel(R) Gigabit Ethernet Network Con= nection > >> [ 343.777900] igb 0000:01:00.2: eth0: (PCIe:5.0Gb/s:Width x4) 00:12:c= 0:00:e9:16 > >> [ 343.777974] igb 0000:01:00.2: eth0: PBA No: 106100-000 > >> [ 343.777977] igb 0000:01:00.2: Using MSI-X interrupts. 4 rx queue(s)= , 4 tx queue(s) > >> [ 343.781905] igb 0000:01:00.2 enp1s0f2: renamed from eth0 > >> [ 343.849424] IPv6: ADDRCONF(NETDEV_UP): enp1s0f2: link is not ready > >> [ 344.139715] igb 0000:01:00.3: added PHC on eth0 > >> [ 344.139718] igb 0000:01:00.3: Intel(R) Gigabit Ethernet Network Con= nection > >> [ 344.139722] igb 0000:01:00.3: eth0: (PCIe:5.0Gb/s:Width x4) 00:12:c= 0:00:e9:17 > >> [ 344.139797] igb 0000:01:00.3: eth0: PBA No: 106100-000 > >> [ 344.139800] igb 0000:01:00.3: Using MSI-X interrupts. 4 rx queue(s)= , 4 tx queue(s) > >> [ 344.141054] igb 0000:01:00.3 enp1s0f3: renamed from eth0 > >> [ 344.213393] IPv6: ADDRCONF(NETDEV_UP): enp1s0f3: link is not ready > >> > >> I googled for this error message and found the following patch: > >> > >> https://www.spinics.net/lists/netdev/msg515557.html > >> > >> I'm not sure if it addresses my problem, but it looks related. I haven= 't > >> tried that yet though because I don't have a spare drive to install > >> Linux and rebuild drivers there. > >> > >> Kevin Bowling wrote: > >> > >>> The phy code hasn't changed in any material way that I can see in a > >>> long time. So I wonder if it's timing or some initialization issue. > >>> > >>> e1000_get_hw_semaphore > >>> e1000_put_hw_semaphore > >>> e1000_read_phy_reg_i2c > >>> > >>> This looks a bit funny, I think the hw sem should be held over the i2c > >>> call. Is there a chance you can get the same log from the working > >>> kernel? > >>> > >>> Regards, > >>> Kevin > >>> > >>> On Wed, Aug 22, 2018 at 2:08 AM, Roman Bogorodskiy > >>> wrote: > >>> > Kevin Bowling wrote: > >>> > > >>> >> Roman, > >>> >> > >>> >> Interesting.. can you set DBG to 1 sys/dev/e1000/e1000_osdep.h Line > >>> >> 109 so we can see closer what is failing to initialize? > >>> > > >>> > Hi Kevin, > >>> > > >>> > Here's a relevant bit of dmesg with debug enabled: > >>> > > >>> > igb1: port 0xe040-0x= e05f mem 0xf7800000-0xf79fffff,0xf7c08000-0xf7c0bfff irq 17 at device 0.1 o= n pci1 > >>> > e1000_set_mac_type > >>> > igb1: attach_pre capping queues at 8 > >>> > e1000_set_mac_type > >>> > e1000_init_mac_ops_generic > >>> > e1000_init_phy_ops_generic > >>> > e1000_init_nvm_ops_generic > >>> > e1000_init_function_pointers_82575 > >>> > e1000_init_mac_params_82575 > >>> > e1000_read_sfp_data_byte > >>> > e1000_read_sfp_data_byte > >>> > e1000_init_nvm_params_82575 > >>> > e1000_init_phy_params_82575 > >>> > e1000_reset_mdicnfg_82580 > >>> > e1000_sgmii_uses_mdio_82575 > >>> > e1000_get_phy_id_82575 > >>> > e1000_sgmii_uses_mdio_82575 > >>> > e1000_read_phy_reg_sgmii_82575 > >>> > e1000_acquire_phy_82575 > >>> > e1000_acquire_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > e1000_read_phy_reg_i2c > >>> > I2CCMD Error bit set > >>> > e1000_release_phy_82575 > >>> > e1000_release_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > PHY address 1 was unreadable > >>> > e1000_read_phy_reg_sgmii_82575 > >>> > e1000_acquire_phy_82575 > >>> > e1000_acquire_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > e1000_read_phy_reg_i2c > >>> > I2CCMD Error bit set > >>> > e1000_release_phy_82575 > >>> > e1000_release_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > PHY address 2 was unreadable > >>> > e1000_read_phy_reg_sgmii_82575 > >>> > e1000_acquire_phy_82575 > >>> > e1000_acquire_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > e1000_read_phy_reg_i2c > >>> > I2CCMD Error bit set > >>> > e1000_release_phy_82575 > >>> > e1000_release_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > PHY address 3 was unreadable > >>> > e1000_read_phy_reg_sgmii_82575 > >>> > e1000_acquire_phy_82575 > >>> > e1000_acquire_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > e1000_read_phy_reg_i2c > >>> > I2CCMD Error bit set > >>> > e1000_release_phy_82575 > >>> > e1000_release_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > PHY address 4 was unreadable > >>> > e1000_read_phy_reg_sgmii_82575 > >>> > e1000_acquire_phy_82575 > >>> > e1000_acquire_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > e1000_read_phy_reg_i2c > >>> > I2CCMD Error bit set > >>> > e1000_release_phy_82575 > >>> > e1000_release_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > PHY address 5 was unreadable > >>> > e1000_read_phy_reg_sgmii_82575 > >>> > e1000_acquire_phy_82575 > >>> > e1000_acquire_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > e1000_read_phy_reg_i2c > >>> > I2CCMD Error bit set > >>> > e1000_release_phy_82575 > >>> > e1000_release_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > PHY address 6 was unreadable > >>> > e1000_read_phy_reg_sgmii_82575 > >>> > e1000_acquire_phy_82575 > >>> > e1000_acquire_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > e1000_read_phy_reg_i2c > >>> > I2CCMD Error bit set > >>> > e1000_release_phy_82575 > >>> > e1000_release_swfw_sync > >>> > e1000_get_hw_semaphore > >>> > e1000_put_hw_semaphore > >>> > PHY address 7 was unreadable > >>> > PHY Initialization Error > >>> > igb1: Setup of Shared code failed, error -2 > >>> > igb1: IFDI_ATTACH_PRE failed 6 > >>> > device_attach: igb1 attach returned 6 > >>> > > >>> > I've also attached a complete dmesg. > >>> > > >>> >> Regards, > >>> >> Kevin > >>> >> > >>> >> On Sun, Aug 19, 2018 at 7:21 AM, Roman Bogorodskiy wrote: > >>> >> > Hi, > >>> >> > > >>> >> > I have a 4-port Intel I350 network card. When I plug sfp rj45 mo= dule to > >>> >> > one of the ports, devices fails to initialise, e.g. ifconfig(8) = only > >>> >> > shows three igb interfaces (expected to be four): > >>> >> > > >>> >> > igb0: flags=3D8802 metric 0 mtu 1500 > >>> >> > options=3De505bb > >>> >> > ether 00:12:c0:00:e9:14 > >>> >> > media: Ethernet autoselect > >>> >> > status: no carrier > >>> >> > nd6 options=3D29 > >>> >> > igb1: flags=3D8802 metric 0 mtu 1500 > >>> >> > options=3De505bb > >>> >> > ether 00:12:c0:00:e9:16 > >>> >> > media: Ethernet autoselect > >>> >> > status: no carrier > >>> >> > nd6 options=3D29 > >>> >> > igb2: flags=3D8802 metric 0 mtu 1500 > >>> >> > options=3De505bb > >>> >> > ether 00:12:c0:00:e9:17 > >>> >> > media: Ethernet autoselect > >>> >> > status: no carrier > >>> >> > nd6 options=3D29 > >>> >> > > >>> >> > dmesg has the following: > >>> >> > > >>> >> > igb0: port 0xe060= -0xe07f mem 0xf7a00000-0xf7bfffff,0xf7c0c000-0xf7c0ffff irq 16 at device 0.= 0 on pci1 > >>> >> > igb0: attach_pre capping queues at 8 > >>> >> > igb0: using 1024 tx descriptors and 1024 rx descriptors > >>> >> > igb0: msix_init qsets capped at 8 > >>> >> > igb0: pxm cpus: 4 queue msgs: 9 admincnt: 1 > >>> >> > igb0: using 4 rx queues 4 tx queues > >>> >> > igb0: Using MSIX interrupts with 5 vectors > >>> >> > igb0: allocated for 4 tx_queues > >>> >> > igb0: allocated for 4 rx_queues > >>> >> > igb0: Ethernet address: 00:12:c0:00:e9:14 > >>> >> > igb0: netmap queues/slots: TX 4/1024, RX 4/1024 > >>> >> > igb1: port 0xe040= -0xe05f mem 0xf7800000-0xf79fffff,0xf7c08000-0xf7c0bfff irq 17 at device 0.= 1 on pci1 > >>> >> > igb1: attach_pre capping queues at 8 > >>> >> > igb1: Setup of Shared code failed, error -2 > >>> >> > igb1: IFDI_ATTACH_PRE failed 6 > >>> >> > device_attach: igb1 attach returned 6 > >>> >> > igb1: port 0xe020= -0xe03f mem 0xf7600000-0xf77fffff,0xf7c04000-0xf7c07fff irq 18 at device 0.= 2 on pci1 > >>> >> > igb1: attach_pre capping queues at 8 > >>> >> > igb1: using 1024 tx descriptors and 1024 rx descriptors > >>> >> > igb1: msix_init qsets capped at 8 > >>> >> > igb1: pxm cpus: 4 queue msgs: 9 admincnt: 1 > >>> >> > igb1: using 4 rx queues 4 tx queues > >>> >> > igb1: Using MSIX interrupts with 5 vectors > >>> >> > igb1: allocated for 4 tx_queues > >>> >> > igb1: allocated for 4 rx_queues > >>> >> > igb1: Ethernet address: 00:12:c0:00:e9:16 > >>> >> > igb1: netmap queues/slots: TX 4/1024, RX 4/1024 > >>> >> > igb2: port 0xe000= -0xe01f mem 0xf7400000-0xf75fffff,0xf7c00000-0xf7c03fff irq 19 at device 0.= 3 on pci1 > >>> >> > igb2: attach_pre capping queues at 8 > >>> >> > igb2: using 1024 tx descriptors and 1024 rx descriptors > >>> >> > igb2: msix_init qsets capped at 8 > >>> >> > igb2: pxm cpus: 4 queue msgs: 9 admincnt: 1 > >>> >> > igb2: using 4 rx queues 4 tx queues > >>> >> > igb2: Using MSIX interrupts with 5 vectors > >>> >> > igb2: allocated for 4 tx_queues > >>> >> > igb2: allocated for 4 rx_queues > >>> >> > igb2: Ethernet address: 00:12:c0:00:e9:17 > >>> >> > igb2: netmap queues/slots: TX 4/1024, RX 4/1024 > >>> >> > > >>> >> > And in pciconf(8) it shows as: > >>> >> > > >>> >> > igb0@pci0:1:0:0: class=3D0x020000 card=3D0x00101b6d chip= =3D0x15228086 rev=3D0x01 hdr=3D0x00 > >>> >> > vendor =3D 'Intel Corporation' > >>> >> > device =3D 'I350 Gigabit Fiber Network Connection' > >>> >> > class =3D network > >>> >> > subclass =3D ethernet > >>> >> > none1@pci0:1:0:1: class=3D0x020000 card=3D0x00101b6d chip= =3D0x15228086 rev=3D0x01 hdr=3D0x00 > >>> >> > vendor =3D 'Intel Corporation' > >>> >> > device =3D 'I350 Gigabit Fiber Network Connection' > >>> >> > class =3D network > >>> >> > subclass =3D ethernet > >>> >> > igb1@pci0:1:0:2: class=3D0x020000 card=3D0x00101b6d chip= =3D0x15228086 rev=3D0x01 hdr=3D0x00 > >>> >> > vendor =3D 'Intel Corporation' > >>> >> > device =3D 'I350 Gigabit Fiber Network Connection' > >>> >> > class =3D network > >>> >> > subclass =3D ethernet > >>> >> > igb2@pci0:1:0:3: class=3D0x020000 card=3D0x00101b6d chip= =3D0x15228086 rev=3D0x01 hdr=3D0x00 > >>> >> > vendor =3D 'Intel Corporation' > >>> >> > device =3D 'I350 Gigabit Fiber Network Connection' > >>> >> > class =3D network > >>> >> > subclass =3D ethernet > >>> >> > > >>> >> > Any suggestions what could be wrong with that? > >>> >> > > >>> >> > The host is FreeBSD 12.0-ALPHA1 amd64 @ r337885. I don't know ho= w/if it > >>> >> > worked before, just plugged that in. > >>> >> > > >>> >> > Roman Bogorodskiy > >>> > > >>> > Roman Bogorodskiy > >> > >> Roman Bogorodskiy Roman Bogorodskiy --s2ZSL+KKDSLx8OML Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJbfmd0AAoJEMltX/4IwiJqGtQH/jS5cNFLKaX1dacF0nvUm/3g gfN9Pxkz7xXBJYcOb6DU50tRWSSNvpvFFt+KSdFKIovX1yQXrFZnYrVrjTU6tWYY N9SnXRlNsTIXPsrAYr4IYPJ3+Mf+m3LbUoxUF4GFWap2OvgB/TtUtmqZGCJC/HFR ZdKbLQSxSmy2LRHWv18iDDGWEntytsQZ0aqLyCDXdHh2J7dr94UzcT6jDDmuhnWy eukR/nkoO6V/yXPVVSJ5ne52p7Rv8ZUqk9eU5533JJX4J7/oHDGq71rCtlKoHAbt +9CW7xgtgecM7JyH8hVcbYgOuIGQ0rZjoiKa+jrkyt4SpCJ+PlVKghorPHnFz0U= =QlFE -----END PGP SIGNATURE----- --s2ZSL+KKDSLx8OML--