From owner-freebsd-net@freebsd.org Thu Aug 23 07:33:33 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50B78108455E for ; Thu, 23 Aug 2018 07:33:33 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id DCEEA85253 for ; Thu, 23 Aug 2018 07:33:32 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by mailman.ysv.freebsd.org (Postfix) id A1988108455D; Thu, 23 Aug 2018 07:33:32 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 56D9E108455C for ; Thu, 23 Aug 2018 07:33:32 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-io0-x243.google.com (mail-io0-x243.google.com [IPv6:2607:f8b0:4001:c06::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C998B85252 for ; Thu, 23 Aug 2018 07:33:31 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by mail-io0-x243.google.com with SMTP id q5-v6so342756iop.3 for ; Thu, 23 Aug 2018 00:33:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Icl1rgEQ9OHN3mMUXA6/R67F/q+DL5IxOO1TaBTfzOk=; b=PKYHPoRl4uae7/4qiRd/jsCuSI8am5QC7XS+CerbbmCnZrqjvyvW1pVbzotVp2Yngt iLZXvZTUrq94u4OtLgK84dx/Ptrb7/wJGumpW5IXrVFNrRQx9oCh40hH+2o8NMLWRmCt HH+/ZdlZKrYLnaLzFL5Q/ZBQM/uzgnd5txlq8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Icl1rgEQ9OHN3mMUXA6/R67F/q+DL5IxOO1TaBTfzOk=; b=XCi13wBqvuMoLxjfbS+uHPeoKZaIY6CixADLIsgbQt56wgibRoCCfcyc1+Xqpu9hDn nEeiVakWteYATuzKn8n9UGDcnFxWi5jPK2w31F+46ywtUIud4oOJk0B/0BHdbO8VH3Bt orIyTudQljMpTU0BOZycw5BzPaJ1h/VLIrCgvLa3Slzv4n/1Y+zVbgL6IQuqjxCEfZSx frl03yky7J8zVv9CSfzVCLkGyddbbvKe09a4R/r3ZGqBSyxPJiZjsU0auLK41cEWphPg yzLpRKOek4gwfsDIDJgzE/+mN0cbWNl0YwpoVQHdTKbyfynVRPdiNX7AbyGKs61b85Wv 6CNQ== X-Gm-Message-State: AOUpUlEO5WmyD68s4ojIs/R5zSRvPfr+eBZF5YP7WPY+ywd3TZu+R7QI HuXgTr/vR5jqgps29uinJtvbIIXj6jaq2Ur82saPRg== X-Google-Smtp-Source: AA+uWPxEtClacMHJ+fO2d07Ds3FQIvBvWJzyE1Lo7pj/sWcwIckm35xzbcnd9UHbAF22z2PygvpyrQ6PxPFziMkgKfA= X-Received: by 2002:a6b:bbc1:: with SMTP id l184-v6mr47074033iof.83.1535009611115; Thu, 23 Aug 2018 00:33:31 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a02:665b:0:0:0:0:0 with HTTP; Thu, 23 Aug 2018 00:33:30 -0700 (PDT) In-Reply-To: References: <20180819142106.GA1539@kloomba> <20180822090811.GA1691@kloomba> <20180823071626.GA4446@kloomba> From: Kevin Bowling Date: Thu, 23 Aug 2018 00:33:30 -0700 Message-ID: Subject: Re: igb: device_attach: igb1 attach returned 6 To: Roman Bogorodskiy Cc: net@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2018 07:33:33 -0000 Well I took a look at the phy code and actually don't see that we abosrbed those commits. I'll try and audit e1000 vs linux in the next 24 hours but in the mean time if you could confirm linux-next or the like fix or don't fix the phy we can at least know if this is the right path or if we need to ask Intel networking for help. Regards, On Thu, Aug 23, 2018 at 12:19 AM, Kevin Bowling wrote: > Great thanks for the info about it not working on Ubuntu too, that > narrows down the problem space considerably. > > Can you try that patch on FreeBSD -CURRENT? It should apply with > minimal changes the phy code is shared by multiple OSes. If it works > we can commit. > > Regards, > > On Thu, Aug 23, 2018 at 12:16 AM, Roman Bogorodskiy > wrote: >> I haven't used this hardware before, so I don't if it ever worked. I >> only tried 11.2 and it doesn't work. I've also tried Ubuntu 18.04 and it >> doesn't work as well. Ubuntu prints the following error: >> >> [ 343.053031] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k >> [ 343.053032] igb: Copyright (c) 2007-2014 Intel Corporation. >> [ 343.415346] igb 0000:01:00.0: added PHC on eth0 >> [ 343.415349] igb 0000:01:00.0: Intel(R) Gigabit Ethernet Network Connection >> [ 343.415352] igb 0000:01:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 00:12:c0:00:e9:14 >> [ 343.415427] igb 0000:01:00.0: eth0: PBA No: 106100-000 >> [ 343.415430] igb 0000:01:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s) >> [ 343.416422] igb 0000:01:00.0 enp1s0f0: renamed from eth0 >> [ 343.416827] igb: probe of 0000:01:00.1 failed with error -3 >> [ 343.485242] IPv6: ADDRCONF(NETDEV_UP): enp1s0f0: link is not ready >> [ 343.777894] igb 0000:01:00.2: added PHC on eth0 >> [ 343.777897] igb 0000:01:00.2: Intel(R) Gigabit Ethernet Network Connection >> [ 343.777900] igb 0000:01:00.2: eth0: (PCIe:5.0Gb/s:Width x4) 00:12:c0:00:e9:16 >> [ 343.777974] igb 0000:01:00.2: eth0: PBA No: 106100-000 >> [ 343.777977] igb 0000:01:00.2: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s) >> [ 343.781905] igb 0000:01:00.2 enp1s0f2: renamed from eth0 >> [ 343.849424] IPv6: ADDRCONF(NETDEV_UP): enp1s0f2: link is not ready >> [ 344.139715] igb 0000:01:00.3: added PHC on eth0 >> [ 344.139718] igb 0000:01:00.3: Intel(R) Gigabit Ethernet Network Connection >> [ 344.139722] igb 0000:01:00.3: eth0: (PCIe:5.0Gb/s:Width x4) 00:12:c0:00:e9:17 >> [ 344.139797] igb 0000:01:00.3: eth0: PBA No: 106100-000 >> [ 344.139800] igb 0000:01:00.3: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s) >> [ 344.141054] igb 0000:01:00.3 enp1s0f3: renamed from eth0 >> [ 344.213393] IPv6: ADDRCONF(NETDEV_UP): enp1s0f3: link is not ready >> >> I googled for this error message and found the following patch: >> >> https://www.spinics.net/lists/netdev/msg515557.html >> >> I'm not sure if it addresses my problem, but it looks related. I haven't >> tried that yet though because I don't have a spare drive to install >> Linux and rebuild drivers there. >> >> Kevin Bowling wrote: >> >>> The phy code hasn't changed in any material way that I can see in a >>> long time. So I wonder if it's timing or some initialization issue. >>> >>> e1000_get_hw_semaphore >>> e1000_put_hw_semaphore >>> e1000_read_phy_reg_i2c >>> >>> This looks a bit funny, I think the hw sem should be held over the i2c >>> call. Is there a chance you can get the same log from the working >>> kernel? >>> >>> Regards, >>> Kevin >>> >>> On Wed, Aug 22, 2018 at 2:08 AM, Roman Bogorodskiy >>> wrote: >>> > Kevin Bowling wrote: >>> > >>> >> Roman, >>> >> >>> >> Interesting.. can you set DBG to 1 sys/dev/e1000/e1000_osdep.h Line >>> >> 109 so we can see closer what is failing to initialize? >>> > >>> > Hi Kevin, >>> > >>> > Here's a relevant bit of dmesg with debug enabled: >>> > >>> > igb1: port 0xe040-0xe05f mem 0xf7800000-0xf79fffff,0xf7c08000-0xf7c0bfff irq 17 at device 0.1 on pci1 >>> > e1000_set_mac_type >>> > igb1: attach_pre capping queues at 8 >>> > e1000_set_mac_type >>> > e1000_init_mac_ops_generic >>> > e1000_init_phy_ops_generic >>> > e1000_init_nvm_ops_generic >>> > e1000_init_function_pointers_82575 >>> > e1000_init_mac_params_82575 >>> > e1000_read_sfp_data_byte >>> > e1000_read_sfp_data_byte >>> > e1000_init_nvm_params_82575 >>> > e1000_init_phy_params_82575 >>> > e1000_reset_mdicnfg_82580 >>> > e1000_sgmii_uses_mdio_82575 >>> > e1000_get_phy_id_82575 >>> > e1000_sgmii_uses_mdio_82575 >>> > e1000_read_phy_reg_sgmii_82575 >>> > e1000_acquire_phy_82575 >>> > e1000_acquire_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > e1000_read_phy_reg_i2c >>> > I2CCMD Error bit set >>> > e1000_release_phy_82575 >>> > e1000_release_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > PHY address 1 was unreadable >>> > e1000_read_phy_reg_sgmii_82575 >>> > e1000_acquire_phy_82575 >>> > e1000_acquire_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > e1000_read_phy_reg_i2c >>> > I2CCMD Error bit set >>> > e1000_release_phy_82575 >>> > e1000_release_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > PHY address 2 was unreadable >>> > e1000_read_phy_reg_sgmii_82575 >>> > e1000_acquire_phy_82575 >>> > e1000_acquire_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > e1000_read_phy_reg_i2c >>> > I2CCMD Error bit set >>> > e1000_release_phy_82575 >>> > e1000_release_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > PHY address 3 was unreadable >>> > e1000_read_phy_reg_sgmii_82575 >>> > e1000_acquire_phy_82575 >>> > e1000_acquire_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > e1000_read_phy_reg_i2c >>> > I2CCMD Error bit set >>> > e1000_release_phy_82575 >>> > e1000_release_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > PHY address 4 was unreadable >>> > e1000_read_phy_reg_sgmii_82575 >>> > e1000_acquire_phy_82575 >>> > e1000_acquire_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > e1000_read_phy_reg_i2c >>> > I2CCMD Error bit set >>> > e1000_release_phy_82575 >>> > e1000_release_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > PHY address 5 was unreadable >>> > e1000_read_phy_reg_sgmii_82575 >>> > e1000_acquire_phy_82575 >>> > e1000_acquire_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > e1000_read_phy_reg_i2c >>> > I2CCMD Error bit set >>> > e1000_release_phy_82575 >>> > e1000_release_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > PHY address 6 was unreadable >>> > e1000_read_phy_reg_sgmii_82575 >>> > e1000_acquire_phy_82575 >>> > e1000_acquire_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > e1000_read_phy_reg_i2c >>> > I2CCMD Error bit set >>> > e1000_release_phy_82575 >>> > e1000_release_swfw_sync >>> > e1000_get_hw_semaphore >>> > e1000_put_hw_semaphore >>> > PHY address 7 was unreadable >>> > PHY Initialization Error >>> > igb1: Setup of Shared code failed, error -2 >>> > igb1: IFDI_ATTACH_PRE failed 6 >>> > device_attach: igb1 attach returned 6 >>> > >>> > I've also attached a complete dmesg. >>> > >>> >> Regards, >>> >> Kevin >>> >> >>> >> On Sun, Aug 19, 2018 at 7:21 AM, Roman Bogorodskiy wrote: >>> >> > Hi, >>> >> > >>> >> > I have a 4-port Intel I350 network card. When I plug sfp rj45 module to >>> >> > one of the ports, devices fails to initialise, e.g. ifconfig(8) only >>> >> > shows three igb interfaces (expected to be four): >>> >> > >>> >> > igb0: flags=8802 metric 0 mtu 1500 >>> >> > options=e505bb >>> >> > ether 00:12:c0:00:e9:14 >>> >> > media: Ethernet autoselect >>> >> > status: no carrier >>> >> > nd6 options=29 >>> >> > igb1: flags=8802 metric 0 mtu 1500 >>> >> > options=e505bb >>> >> > ether 00:12:c0:00:e9:16 >>> >> > media: Ethernet autoselect >>> >> > status: no carrier >>> >> > nd6 options=29 >>> >> > igb2: flags=8802 metric 0 mtu 1500 >>> >> > options=e505bb >>> >> > ether 00:12:c0:00:e9:17 >>> >> > media: Ethernet autoselect >>> >> > status: no carrier >>> >> > nd6 options=29 >>> >> > >>> >> > dmesg has the following: >>> >> > >>> >> > igb0: port 0xe060-0xe07f mem 0xf7a00000-0xf7bfffff,0xf7c0c000-0xf7c0ffff irq 16 at device 0.0 on pci1 >>> >> > igb0: attach_pre capping queues at 8 >>> >> > igb0: using 1024 tx descriptors and 1024 rx descriptors >>> >> > igb0: msix_init qsets capped at 8 >>> >> > igb0: pxm cpus: 4 queue msgs: 9 admincnt: 1 >>> >> > igb0: using 4 rx queues 4 tx queues >>> >> > igb0: Using MSIX interrupts with 5 vectors >>> >> > igb0: allocated for 4 tx_queues >>> >> > igb0: allocated for 4 rx_queues >>> >> > igb0: Ethernet address: 00:12:c0:00:e9:14 >>> >> > igb0: netmap queues/slots: TX 4/1024, RX 4/1024 >>> >> > igb1: port 0xe040-0xe05f mem 0xf7800000-0xf79fffff,0xf7c08000-0xf7c0bfff irq 17 at device 0.1 on pci1 >>> >> > igb1: attach_pre capping queues at 8 >>> >> > igb1: Setup of Shared code failed, error -2 >>> >> > igb1: IFDI_ATTACH_PRE failed 6 >>> >> > device_attach: igb1 attach returned 6 >>> >> > igb1: port 0xe020-0xe03f mem 0xf7600000-0xf77fffff,0xf7c04000-0xf7c07fff irq 18 at device 0.2 on pci1 >>> >> > igb1: attach_pre capping queues at 8 >>> >> > igb1: using 1024 tx descriptors and 1024 rx descriptors >>> >> > igb1: msix_init qsets capped at 8 >>> >> > igb1: pxm cpus: 4 queue msgs: 9 admincnt: 1 >>> >> > igb1: using 4 rx queues 4 tx queues >>> >> > igb1: Using MSIX interrupts with 5 vectors >>> >> > igb1: allocated for 4 tx_queues >>> >> > igb1: allocated for 4 rx_queues >>> >> > igb1: Ethernet address: 00:12:c0:00:e9:16 >>> >> > igb1: netmap queues/slots: TX 4/1024, RX 4/1024 >>> >> > igb2: port 0xe000-0xe01f mem 0xf7400000-0xf75fffff,0xf7c00000-0xf7c03fff irq 19 at device 0.3 on pci1 >>> >> > igb2: attach_pre capping queues at 8 >>> >> > igb2: using 1024 tx descriptors and 1024 rx descriptors >>> >> > igb2: msix_init qsets capped at 8 >>> >> > igb2: pxm cpus: 4 queue msgs: 9 admincnt: 1 >>> >> > igb2: using 4 rx queues 4 tx queues >>> >> > igb2: Using MSIX interrupts with 5 vectors >>> >> > igb2: allocated for 4 tx_queues >>> >> > igb2: allocated for 4 rx_queues >>> >> > igb2: Ethernet address: 00:12:c0:00:e9:17 >>> >> > igb2: netmap queues/slots: TX 4/1024, RX 4/1024 >>> >> > >>> >> > And in pciconf(8) it shows as: >>> >> > >>> >> > igb0@pci0:1:0:0: class=0x020000 card=0x00101b6d chip=0x15228086 rev=0x01 hdr=0x00 >>> >> > vendor = 'Intel Corporation' >>> >> > device = 'I350 Gigabit Fiber Network Connection' >>> >> > class = network >>> >> > subclass = ethernet >>> >> > none1@pci0:1:0:1: class=0x020000 card=0x00101b6d chip=0x15228086 rev=0x01 hdr=0x00 >>> >> > vendor = 'Intel Corporation' >>> >> > device = 'I350 Gigabit Fiber Network Connection' >>> >> > class = network >>> >> > subclass = ethernet >>> >> > igb1@pci0:1:0:2: class=0x020000 card=0x00101b6d chip=0x15228086 rev=0x01 hdr=0x00 >>> >> > vendor = 'Intel Corporation' >>> >> > device = 'I350 Gigabit Fiber Network Connection' >>> >> > class = network >>> >> > subclass = ethernet >>> >> > igb2@pci0:1:0:3: class=0x020000 card=0x00101b6d chip=0x15228086 rev=0x01 hdr=0x00 >>> >> > vendor = 'Intel Corporation' >>> >> > device = 'I350 Gigabit Fiber Network Connection' >>> >> > class = network >>> >> > subclass = ethernet >>> >> > >>> >> > Any suggestions what could be wrong with that? >>> >> > >>> >> > The host is FreeBSD 12.0-ALPHA1 amd64 @ r337885. I don't know how/if it >>> >> > worked before, just plugged that in. >>> >> > >>> >> > Roman Bogorodskiy >>> > >>> > Roman Bogorodskiy >> >> Roman Bogorodskiy