From owner-freebsd-bugs@freebsd.org Tue Jul 28 13:45:18 2020 Return-Path: Delivered-To: freebsd-bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BC5D436AA14 for ; Tue, 28 Jul 2020 13:45:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4BGHxB4czqz4RXd for ; Tue, 28 Jul 2020 13:45:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 9EABE36A998; Tue, 28 Jul 2020 13:45:18 +0000 (UTC) Delivered-To: bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9E6DC36A264 for ; Tue, 28 Jul 2020 13:45:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BGHxB3mMDz4RSH for ; Tue, 28 Jul 2020 13:45:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 66271FB62 for ; Tue, 28 Jul 2020 13:45:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 06SDjIFq068357 for ; Tue, 28 Jul 2020 13:45:18 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 06SDjIYR068356 for bugs@FreeBSD.org; Tue, 28 Jul 2020 13:45:18 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 248324] qlxge: stability issues (unusable) with QLogic QLE8142 NIC Date: Tue, 28 Jul 2020 13:45:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: Nyakov13@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jul 2020 13:45:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D248324 Bug ID: 248324 Summary: qlxge: stability issues (unusable) with QLogic QLE8142 NIC Product: Base System Version: 12.1-RELEASE Hardware: amd64 OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: Nyakov13@gmail.com Short: QLogic QLE8142 Convergent network adapters do not work properly on FreeBSD (10.2 and 12.1 tested). Low speeds and connection dropouts. But they work perfectly with Linux (qlge driver) I was suggested that this can be related to mismatched mtu size somewhere. I also suspect that this somehow can be related to flowcontrol and media detection. Long: I have two 10G Qlogic converged network cards. QLogic QLE8142-IBMX (42C1802) QLogic QLE8142-SR-IBM (46K8088) They are basically the same(looks and works same), both flashed with latest firmware (from Qlogic support) (other firmware versions have no effects on unstable behavior) They are transceiver specific and I use them with Qlogic FLTX8571D3BCL-QL optical transceivers (it seems they support only fiber) I am testing them with iperf3. I setup direct fiber link between two NICs on different PCs with different = OS Like this (ifconfig ql0 inet 192.168.10.1/24) and (192.168.10.2/24) for sec= ond PC. On FreeBSD this card got identified as: ql0: port 0xe300-0xe3ff mem 0xfea0c000-0xfea0ffff,0xfe800000-0xfe8fffff at device 0.0= on pci1 ql0:=20 qls_pci_attach: ha 0xfffffe0000696000 pci_func 0x0 msix_count 0x1 pci_reg 0xfffff80003720700 pci_reg1 0xfffff80003720600 ql0: Ethernet address: 00:c0:dd:26:25:80 On Linux: qlge 0000:01:00.0: QLogic 10 Gigabit PCI-E Ethernet Driver=20 qlge 0000:01:00.0: Driver name: qlge, Version: 1.00.00.35. qlge 0000:01:00.0 eth1: Link is down. qlge 0000:01:00.0 eth1: Clearing MAC address qlge 0000:01:00.0 eth1: Function #0, Port 0, NIC Roll 0, NIC Rev =3D 1, XG = Roll =3D 0, XG R> qlge 0000:01:00.0 eth1: MAC address 00:c0:dd:26:25:80 What works: OS Linux (Arch and Debian tested) qlge (https://cateee.net/lkddb/web-lkddb/QLGE.html) kernel module. QLE8142 -> fiber -> any transceiver(including Qlogic one) + any 10G card at= any os(except QLE8142 card on FreeBSD) mtu 9000 and mtu 1500 All great, I see channel saturation and no errors. Different offload settin= gs have small effect on anything. What does not work: QLE8142 card on FreeBSD with any card in other end (including QLE8142 on Li= nux) OS FreeBSD qlxge (https://www.freebsd.org/cgi/man.cgi?query=3Dqlxge) kernel module, FreeBSD10.2 and FreeBSD12.1 tested. I get unstable behavior. Low link speed, or dropout of link when test start. It is unstable, so, it is hard to describe whats going on. One example: iperf3 connecting from Linux box to FreeBSD12.1 with QLE8142 - all seems fi= ne. iperf3 connecting from FreeBSD12.1 with QLE8142 to Linux box - low unstable speed (2Gb/s - 5Gb/s) then dropout to zero. Link seems to stay active but no new connection can pass at all(for some time) This somehow gets affected by -tso setting, but unstable behavior just beco= me different. On FreeBSD10.2 with -tso setting and mtu 1500 - card can work stable for so= me time, but sometimes it is not. With mtu 9000 - unstable, drop of connections. Another example: Testing two direct connected QLE8142 cards, one on Linux other on FreeBSD12= .1 All default. mtu 1500. I get 8.2Gb/s from Linux to BSD, 1Gb/s from BSD to Linux. When on FreeBSD I disable TSO, I get 5Gb/s From BSD to Linux. And 4Gb/s in both directions simultaneously. Then. I set mtu 9000 on both ends. >From Linux to BSD - drop of connection. >From BSD to Linux 2Gb/s mtu 9000 and disable TSO on FreeBSD end >From Linux to BSD - drop of connection. >From BSD to Linux 9,5Gb/s FreeBSD show messages in log such as: ql0: qls_mbx_get_link_status 0x00004000 0x10000051 0x00000000 0x00000037 0x000000f9 0x05050504 ql0: qls_hw_send: tx_free[0] =3D 2 ql0: qls_mbx_get_link_status 0x00004000 0x10000051 0x00000000 0x00000037 0x000000fd 0x05050504 ql0: qls_mbx_isr: AEN [0x00008012 0x00000060 0x00000037 0x00000000 0x000000= 00 0x00000000 0x00000000 0x00000000 0x00000000] ql0: qls_rx_comp: DS bit not set=20 ql0: qls_rx_comp: (rxb->paddr !=3D cq_e->b_paddr)[0x55270000, 0x5527c800]=20 ql0: qls_rx_comp: (rxb->paddr !=3D cq_e->b_paddr)[0x55270000, 0x35b3cc000]= =20 ql0: qls_rx_comp: (rxb->paddr !=3D cq_e->b_paddr)[0x55270000, 0x53fe4000] FreeBSD ifconfig -m ql0 output: ql0: flags=3D8843 metric 0 mtu 1500 =20=20=20=20=20=20=20 options=3Dc013b =20=20=20=20=20=20=20 capabilities=3Dc013b ether 00:c0:dd:26:25:80 inet 192.168.10.1 netmask 0xffffff00 broadcast 192.168.10.255 media: Ethernet autoselect (10Gbase-SR ) status: active supported media: media autoselect media autoselect mediaopt full-duplex nd6 options=3D29 >From Linux: sudo ethtool enp1s0f0 netlink error: No such file or directory Settings for enp1s0f0: Supported ports: [ FIBRE ] Supported link modes: 10000baseT/Full=20 Supported pause frame use: No Supports auto-negotiation: No Supported FEC modes: Not reported Advertised link modes: 10000baseT/Full=20 Advertised pause frame use: No Advertised auto-negotiation: No Advertised FEC modes: Not reported Speed: 10000Mb/s Duplex: Full Port: FIBRE PHYAD: 0 Transceiver: internal Auto-negotiation: off Supports Wake-on: d Wake-on: d Current message level: 0x000060f7 (24823) drv probe link ifdown ifup rx_err tx_err hw = wol I can provide ssh access to test environment (email me) --=20 You are receiving this mail because: You are the assignee for the bug.=