From nobody Fri Jan 14 10:50:17 2022 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 29D5A1966773 for ; Fri, 14 Jan 2022 10:50:26 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4JZykS6gB9z4jrS for ; Fri, 14 Jan 2022 10:50:24 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=cs.huji.ac.il; s=57791128; h=To:Date:Message-Id:Subject:Mime-Version:Content-Transfer-Encoding:Content-Type:From; bh=/SzIP/Pp3CeB1g9hiL53ZuPk+By5352yed9NtnqtsGI=; b=ubOA/ziWS+BU0dqlyeACsBC8DYC9k5DVvIoehXubxJt8AdZVYT1Y3ageRdZmopwazlawChG9xdx3Z8iZkkXbpIqR9V4WVIZaV26C+wR9yflePB1ewlZzkB2Q/LwRzDzUi7YZNVRD2aBKqmndBIW/l/mdggADvDktSBGAFlQQVhHb1dFLhxgcwW+pINdMiIZdkWOkCtwu3EC3X6sO7W1vFZqItB4U8ZsW+0YQZyG6apnyhL6/hIJvSEB+JNpbgDVzUpf44eA5sKeXRkIPNgL1SMbvC2bpRXnExu//V+cAs2zfiI7xeFLueGbmna+4MAtvMlFgJHIkgy4s+C1S0YYDSQ==; Received: from bach.cs.huji.ac.il ([132.65.80.20] helo=smtpclient.apple) by kabab.cs.huji.ac.il with esmtp id 1n8K9x-000GkE-EI for freebsd-virtualization@freebsd.org; Fri, 14 Jan 2022 12:50:17 +0200 From: Daniel Braniss Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: bhyve terribly slow when using tagged vlan Message-Id: <878640B2-CB46-4066-8655-20A18D3A6A7D@cs.huji.ac.il> Date: Fri, 14 Jan 2022 12:50:17 +0200 To: freebsd-virtualization@freebsd.org X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4JZykS6gB9z4jrS X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cs.huji.ac.il header.s=57791128 header.b="ubOA/ziW"; dmarc=pass (policy=none) header.from=huji.ac.il; spf=none (mx1.freebsd.org: domain of danny@cs.huji.ac.il has no SPF policy when checking 132.65.116.210) smtp.mailfrom=danny@cs.huji.ac.il X-Spamd-Result: default: False [-2.74 / 15.00]; ARC_NA(0.00)[]; SUBJECT_ENDS_SPACES(0.50)[]; R_DKIM_ALLOW(-0.20)[cs.huji.ac.il:s=57791128]; FREEFALL_USER(0.00)[danny]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-0.95)[-0.947]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_MEDIUM(-0.99)[-0.995]; DKIM_TRACE(0.00)[cs.huji.ac.il:+]; DMARC_POLICY_ALLOW(-0.50)[huji.ac.il,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-virtualization]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:378, ipnet:132.64.0.0/15, country:IL]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N hi, the host is a very new Dell PowerEdge R6525 running FreeBSD 13.0, I have connected ixl0 to an untagged link and ixl1 to a tagged link I have created 2 bhyve clients, identical, except for the links, one is = using ixl0 and the other ixl1.340 the untagged boots diskless in about 40 seconds, while the one using the = tagged one takes 12 minutes! I compared their dmesgs and they are identical (except for minor diffs = like mac address) any ideas? danny some more data: sa# uname -a FreeBSD sa 13.0-STABLE FreeBSD 13.0-STABLE #4 huji-13-736697896: Mon Dec = 13 14:03:04 IST 2021 = danny@hms-00:/home/obj/hms-00/h/rnd/git/stable/13/amd64.amd64/sys/HUJI = amd64 sa# ifconfig ixl0: flags=3D8963 = metric 0 mtu 1500 = options=3D4a500b9 ether b4:96:91:97:08:14 inet 132.65.116.32 netmask 0xfffff000 broadcast 132.65.127.255 media: Ethernet autoselect (10Gbase-T ) status: active nd6 options=3D29 ixl1: flags=3D8963 = metric 0 mtu 1500 = options=3D4e507bb ether b4:96:91:97:08:15 media: Ethernet autoselect (10Gbase-T ) status: active nd6 options=3D29 bge0: flags=3D8802 metric 0 mtu 1500 = options=3Dc019b ether b0:7b:25:dd:ff:9c media: Ethernet autoselect nd6 options=3D29 bge1: flags=3D8802 metric 0 mtu 1500 = options=3Dc019b ether b0:7b:25:dd:ff:9d media: Ethernet autoselect nd6 options=3D29 lo0: flags=3D8049 metric 0 mtu 16384 options=3D680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=3D21 ixl1.340: flags=3D8943 = metric 0 mtu 1500 options=3D4200401 ether b4:96:91:97:08:15 inet 132.65.80.251 netmask 0xfffff000 broadcast 132.65.95.255 groups: vlan vlan: 340 vlanproto: 802.1q vlanpcp: 0 parent interface: ixl1 media: Ethernet autoselect (10Gbase-T ) status: active nd6 options=3D29 ue0: flags=3D8802 metric 0 mtu 1500 ether b0:7b:25:e5:dc:57 nd6 options=3D29 vm-servers: flags=3D8843 metric = 0 mtu 1500 ether 0a:91:de:0e:f6:26 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap0 flags=3D143 ifmaxaddr 0 port 10 priority 128 path cost 2000000 member: ixl0 flags=3D143 ifmaxaddr 0 port 1 priority 128 path cost 2000 groups: bridge vm-switch viid-d5539@ nd6 options=3D9 vm-system: flags=3D8843 metric 0 = mtu 1500 ether 42:5b:f8:8f:72:00 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap1 flags=3D143 ifmaxaddr 0 port 11 priority 128 path = cost 2000000 member: ixl1.340 flags=3D143 ifmaxaddr 0 port 6 priority 128 path cost 2000 groups: bridge vm-switch viid-54b53@ nd6 options=3D9 tap0: flags=3D8943 = metric 0 mtu 1500 description: vmnet-bhv-08-0-servers options=3D80000 ether 58:9c:fc:10:82:13 groups: tap vm-port media: Ethernet autoselect status: active nd6 options=3D29 Opened by PID 9357 tap1: flags=3D8943 = metric 0 mtu 1500 description: vmnet-bhv-09-0-system options=3D80000 ether 58:9c:fc:10:ff:a2 groups: tap vm-port media: Ethernet autoselect status: active nd6 options=3D29=