From owner-freebsd-net@freebsd.org Fri Feb 22 10:31:41 2019 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CFA814E57BC for ; Fri, 22 Feb 2019 10:31:41 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2B1148DE9D for ; Fri, 22 Feb 2019 10:31:34 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate2.intern.punkt.de with ESMTP id x1MAVQub075551 for ; Fri, 22 Feb 2019 11:31:26 +0100 (CET) Received: from [217.29.44.250] ([217.29.44.250]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id x1MAVQY6090075 for ; Fri, 22 Feb 2019 11:31:26 +0100 (CET) (envelope-from hausen@punkt.de) From: "Patrick M. Hausen" Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Performance issues with VNET/bridge/VLAN Message-Id: <9B0EC546-38E6-424E-9CC9-93F4C58B296F@punkt.de> Date: Fri, 22 Feb 2019 11:31:26 +0100 To: FreeBSD Net X-Mailer: Apple Mail (2.3445.9.1) X-Rspamd-Queue-Id: 2B1148DE9D X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 217.29.33.131 as permitted sender) smtp.mailfrom=hausen@punkt.de X-Spamd-Result: default: False [-3.30 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:217.29.32.0/20]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; DMARC_NA(0.00)[punkt.de]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[mailin.pluspunkthosting.de,mailin.pluspunkthosting.de]; NEURAL_HAM_SHORT(-0.95)[-0.954,0]; RCVD_IN_DNSWL_NONE(0.00)[131.33.29.217.list.dnswl.org : 127.0.10.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(-0.54)[ipnet: 217.29.32.0/20(-1.51), asn: 16188(-1.21), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16188, ipnet:217.29.32.0/20, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Feb 2019 10:31:41 -0000 Hi all, please have a look at these two network setups: ------- separate interfaces -------=20 ifconfig_ixl0=3D"up" ifconfig_ixl1=3D"up" cloned_interfaces=3D"bridge0 bridge1" ifconfig_bridge0_name=3D"inet0" ifconfig_inet0=3D"addm ixl0 up" ifconfig_inet0_alias0=3D"inet ww.xx.yy.zz/24" ifconfig_inet0_ipv6=3D"inet6 2a00:b580:8000:11:dead:beef:dead:beef/64 = auto_linklocal" ifconfig_bridge1_name=3D"mgmt0" ifconfig_mgmt0=3D"addm ixl1 up" ifconfig_mgmt0_alias0=3D"inet 10.x.y.z/16" ifconfig_mgmt0_ipv6=3D"inet6 auto_linklocal" -----------------------------------=20 and ------- trunk port w/ VLANs -------=20 ifconfig_ixl0=3D"up" cloned_interfaces=3D"vlan7 vlan11 bridge0 bridge1" ifconfig_vlan7=3D"up vlan 7 vlandev ixl0" ifconfig_vlan11=3D"up vlan 11 vlandev ixl0" ifconfig_bridge0_name=3D"inet0" ifconfig_inet0=3D"addm vlan11 up" ifconfig_inet0_alias0=3D"inet ww.xx.yy.zz/24" ifconfig_inet0_ipv6=3D"inet6 2a00:b580:8000:11:dead:beef:dead:beef/64 = auto_linklocal" ifconfig_bridge1_name=3D"mgmt0" ifconfig_mgmt0=3D"addm vlan7 up" ifconfig_mgmt0_alias0=3D"inet 10.x.y.z/16" ifconfig_mgmt0_ipv6=3D"inet6 auto_linklocal" -----------------------------------=20 If the switches at the other end are configured correctly, they should = work precisely the same, right? In the second case both networks share the = bandwidth, but the management network is mostly idle and only used for monitoring = and nightly backups. The machine is an iocage jail host, all jails with VNET. The problem is: network performance in the jails (not on the host!) is = abysmal with the second setup. Not consistently so, everything *seems* to work but e.g. a customer complained that checking out a project from github happend at 15k/s =E2=80=A6 that=E2=80=99s when we started to = investigate. After not finding anything fundamentally wrong we checked the = differences between individual hosts and the only essential one was the network = setup as shown above. So early this morning i rewired the host, got rid of the VLANs - presto, github fast now. *Any* idea what might be going on here? We use VNET all the same on all = the hosts and it is still labelled =E2=80=9Eexperimental", yes. But all the = parts that make up the different setups - bridge(4), vlan(4) - have been in FreeBSD for ages. I=E2=80=99m just combining features orthogonally like every = good sysadmin ;-) If someone is willing to do some investigation, I think I can provide a = test system and remote access =E2=80=A6 Systems are running 11.2p3 at the moment. To be patched to 11.2p9 next Tuesday. Kind regards, Patric --=20 punkt.de GmbH Internet - Dienstleistungen - Beratung Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100 76133 Karlsruhe info@punkt.de http://punkt.de AG Mannheim 108285 Gf: Juergen Egeling