From nobody Fri Oct 25 20:21:47 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XZvMT260wz5b75Z for ; Fri, 25 Oct 2024 20:21:57 +0000 (UTC) (envelope-from lexi@le-fay.org) Received: from fuchsia.eden.le-Fay.ORG (fuchsia.eden.le-fay.org [81.187.47.195]) by mx1.freebsd.org (Postfix) with ESMTP id 4XZvMS3BXjz4nfb for ; Fri, 25 Oct 2024 20:21:56 +0000 (UTC) (envelope-from lexi@le-fay.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=le-fay.org header.s=fuchsia header.b="BU9/jmyM"; spf=pass (mx1.freebsd.org: domain of lexi@le-fay.org designates 81.187.47.195 as permitted sender) smtp.mailfrom=lexi@le-fay.org; dmarc=none Received: from iris.eden.le-Fay.ORG (unknown [IPv6:2001:8b0:aab5:7::11]) by fuchsia.eden.le-Fay.ORG (Postfix) with ESMTP id 98483501; Fri, 25 Oct 2024 20:21:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=le-fay.org; s=fuchsia; t=1729887708; bh=OQAwOITPAmtbTu66fve3uv9rFGww5Cq0uhQY+GIeKe8=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=BU9/jmyMLyf4tqFCaZ+WBIXVyZJG50fzyZSkv6qXho0ghK3UBrfS6khHA2yOZJzv5 LU/uU/rB1wk62AOuRvJFU88jWzA83Vx5kmQMKL8x1wximVe5oDsrX4n2b3WjcHnD/W BIAcKo2aGENBL1KET9xjvoSM4j3yxs3z3GmS125k= Received: from [IPV6:2001:8b0:aab5:c210:2052:cb63:121f:e82] (unknown [IPv6:2001:8b0:aab5:c210:2052:cb63:121f:e82]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by iris.eden.le-Fay.ORG (Postfix) with ESMTPSA id 591052C0475; Fri, 25 Oct 2024 21:21:47 +0100 (BST) Message-ID: Date: Fri, 25 Oct 2024 21:21:47 +0100 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Chelsio / cxlv(4): strange messages, SR-IOV interface does not work Content-Language: en-GB To: Navdeep Parhar Cc: freebsd-net@freebsd.org References: From: Lexi Winter In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.39 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; R_SPF_ALLOW(-0.20)[+ip4:81.187.47.195]; R_DKIM_ALLOW(-0.20)[le-fay.org:s=fuchsia]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; XM_UA_NO_VERSION(0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:20712, ipnet:81.187.0.0/16, country:GB]; FROM_HAS_DN(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DMARC_NA(0.00)[le-fay.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; DKIM_TRACE(0.00)[le-fay.org:+] X-Rspamd-Queue-Id: 4XZvMS3BXjz4nfb X-Spamd-Bar: --- On 24/10/2024 00:43, Navdeep Parhar wrote: > On Tue, Oct 22, 2024 at 11:21 PM Lexi Winter > wrote: > however, trying to assign an IP address causes immediate problems: > > root@lily:~ # ifconfig cxlv0 inet6 2001:8b0:aab5:7::10/64 > root@lily:~ # Oct 23 06:16:07 lily kernel: cxlv0: a looped back NS > message is detected during DAD for fe80:3::444:3fff:fee7:6030.  Another > DAD probes are being sent. > You can disable IPv6 DAD as a workaround for this issue.  The problem is > that the VF's multicast tx is getting reflected back to it by the > internal switch when it shouldn't. > > # sysctl net.inet6.ip6.dad_count=0 thanks Navdeep, after enabling this, networking is working in the VM. however, i can't tell if VLAN filtering is supposed to be working for VF interfaces. from looking at src, it seems like after a couple of recent commits (c57c261790 and 1389314d5) this should be supported. so i configured a VLAN restriction in the iovctl.conf: PF { device : "cxl3"; num_vfs : 16; } DEFAULT { passthrough : true; vlan : 111; } VF-0 { passthrough : true; vlan : 111; mac-addr : "06:00:00:00:05:a0" } configuring cxlv0.111 in the VM works and the VM can communicate on VLAN 111, but if i look at tcpdump inside the VM, i also see untagged traffic on the host port: 21:16:23.874161 74:4d:28:08:81:f6 > 01:80:c2:00:00:0e, ethertype LLDP (0x88cc), length 177: LLDP, length 163: c1.i.lfns.org.uk as well as traffic for VLAN 3003, which is the bhyve host's VLAN: 21:14:28.550117 06:00:00:00:06:01 > 01:00:5e:00:00:0d, ethertype 802.1Q (0x8100), length 72: vlan 3003, p 0, ethertype IPv4 (0x0800), 172.22.0.37 > 224.0.0.13: PIMv2, Hello, length 34 so, it seems like there's no actual VLAN filtering happening in the hardware. is there a sysctl that needs to be enabled to work this? (i did have a quick look through the hw.cxgbe MIB but didn't see anything obvious.) or is this only supported for T4 ASICs, not T5?