From owner-freebsd-net@freebsd.org Sat Feb 20 00:39:02 2021 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5C4615471A5 for ; Sat, 20 Feb 2021 00:39:02 +0000 (UTC) (envelope-from pen@lysator.liu.se) Received: from ste-pvt-msa1.bahnhof.se (ste-pvt-msa1.bahnhof.se [213.80.101.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dj8hP2LzCz3rv7 for ; Sat, 20 Feb 2021 00:39:01 +0000 (UTC) (envelope-from pen@lysator.liu.se) Received: from localhost (localhost [127.0.0.1]) by ste-pvt-msa1.bahnhof.se (Postfix) with ESMTP id 82A5E408D6 for ; Sat, 20 Feb 2021 01:38:58 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at bahnhof.se X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=6.31 tests=[BAYES_00=-1.9] autolearn=ham autolearn_force=no Received: from ste-pvt-msa1.bahnhof.se ([127.0.0.1]) by localhost (ste-pvt-msa1.bahnhof.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ERe9fDU31YB5 for ; Sat, 20 Feb 2021 01:38:57 +0100 (CET) Received: by ste-pvt-msa1.bahnhof.se (Postfix) with ESMTPA id A21BB4027F for ; Sat, 20 Feb 2021 01:38:57 +0100 (CET) Received: from [192.168.1.132] ([192.168.1.132]) (authenticated bits=0) by hope.grebo.net (8.16.1/8.16.1) with ESMTPSA id 11K0coRN099892 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 20 Feb 2021 01:38:51 +0100 (CET) (envelope-from pen@lysator.liu.se) From: Peter Eriksson Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Intel ixl lagg boot problem on FreeBSD 12.2 on Dell R730xd Message-Id: <27AD8EE0-F5D0-429C-958B-D72680892365@lysator.liu.se> Date: Sat, 20 Feb 2021 01:38:50 +0100 To: "freebsd-net@freebsd.org" X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (hope.grebo.net [10.0.0.3]); Sat, 20 Feb 2021 01:38:51 +0100 (CET) X-Rspamd-Queue-Id: 4Dj8hP2LzCz3rv7 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=liu.se (policy=none); spf=neutral (mx1.freebsd.org: 213.80.101.70 is neither permitted nor denied by domain of pen@lysator.liu.se) smtp.mailfrom=pen@lysator.liu.se X-Spamd-Result: default: False [-2.57 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[liu.se : No valid SPF, No valid DKIM,none]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_NEUTRAL(0.00)[?all]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; RBL_DBL_DONT_QUERY_IPS(0.00)[213.80.101.70:from]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[213.80.101.70:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.970]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8473, ipnet:213.80.96.0/19, country:SE]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-net]; RCVD_IN_DNSWL_LOW(-0.10)[213.80.101.70:from] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2021 00:39:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D253708 Whenever I try to boot a Dell R730xd server with FreeBSD 12.2-RELEASE-p3 = it always locks up on boot - when attempting to enable the second Intel = ixl 10G ethernet interface in a lagg (failover) link. Unless I enable = verbose boot - then it starts up just fine=E2=80=A6. So I=E2=80=99m = guessing it=E2=80=99s a timing issue... Anyone else seeing this?=20 I=E2=80=99ve updated the server with the latest firmwares (BIOS, Intel = cards etc) from Dell. This is the end of the boot (non-verbose) boot output. An interesting = thing is that it also kills the IPMI serial-over-lan session when this = happens. It happens on two R730xd servers (the ones I=E2=80=99ve = upgraded from 11.3/11.4 so far=E2=80=A6). The output always ends after = that ixl2 message. > xhci0: 32 bytes context size, 64-bit DMA > ixl0: Link is up, 10 Gbps Full Duplex, Requested FEC: None, Negotiated = FEC: None, Autoneg: False, Flow Control: None > ixl0: link state changed to UP > usbus0 numa-domain 0 on xhci0 > usbus0: 5.0Gbps Super Speed USB v3.0 > pci2: at device 22.0 (no driver attached) > pci2: at device 22.1 (no driver attached) > ehci0: mem 0x97113000-0x971133ff = at device 26.0 numa-domain 0 on pci2 > usbus1: EHCI version 1.0 > usbus1 numa-domain 0 on ehci0 > usbus1: 480Mbps High Speed USB v2.0 > pcib7: at device 28.0 numa-domain 0 on pci2 > pci7: numa-domain 0 on pcib7 > pcib8: at device 28.7 numa-domain 0 on pci2 > pci8: numa-domain 0 on pcib8 > pcib9: at device 0.0 numa-domain 0 on pci8 > pci9: numa-domain 0 on pcib9 > pcib10: at device 0.0 numa-domain 0 on pci9 > pci10: numa-domain 0 on pcib10 > ixl2: Link is upError: No response to keepalive - Terminating session Anyone else seen this problem? - Peter